PyTorchKR Papers

최신 논문

총 3,337편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2105.01458

Learning-based feedforward augmentation for steady state rejection of residual dynamics on a nanometer-accurate planar actuator system

Growing demands in the semiconductor industry result in the need for enhanced performance of lithographic equipment. However, position tracking accuracy of high precision mechatronics is often limited by the presence of disturbance sources, which originate from unmodelled or unforeseen deterministic environmental effects. To negate the effects of these disturbances, a learning based feedforward controller is employed, where the underlying control policy is estimated from experimental data based on Gaussian Process regression. The proposed approach exploits the property of including prior knowledge on the expected steady state behaviour of residual dynamics in terms of kernel selection. Corresponding hyper-parameters are optimized using the maximization of the marginalized likelihood. Consequently, the learned function is employed as augmentation of the currently employed rigid body feedforward controller. The effectiveness of the augmentation is experimentally validated on a magnetically levitated planar motor stage. The results of this paper demonstrate the benefits and possibilities of machine-learning based approaches for compensation of static effects, which originate from residual dynamics, such that position tracking performance for moving-magnet planar motor actuators is improved.

arXiv:2105.01478

Intelligent Zero Trust Architecture for 5G/6G Networks: Principles, Challenges, and the Role of Machine Learning in the context of O-RAN

In this position paper, we discuss the critical need for integrating zero trust (ZT) principles into next-generation communication networks (5G/6G). We highlight the challenges and introduce the concept of an intelligent zero trust architecture (i-ZTA) as a security framework in 5G/6G networks with untrusted components. While network virtualization, software-defined networking (SDN), and service-based architectures (SBA) are key enablers of 5G networks, operating in an untrusted environment has also become a key feature of the networks. Further, seamless connectivity to a high volume of devices has broadened the attack surface on information infrastructure. Network assurance in a dynamic untrusted environment calls for revolutionary architectures beyond existing static security frameworks. To the best of our knowledge, this is the first position paper that presents the architectural concept design of an i-ZTA upon which modern artificial intelligence (AI) algorithms can be developed to provide information security in untrusted networks. We introduce key ZT principles as real-time Monitoring of the security state of network assets, Evaluating the risk of individual access requests, and Deciding on access authorization using a dynamic trust algorithm, called MED components. To ensure ease of integration, the envisioned architecture adopts an SBA-based design, similar to the 3GPP specification of 5G networks, by leveraging the open radio access network (O-RAN) architecture with appropriate real-time engines and network interfaces for collecting necessary machine learning data. Therefore, this work provides novel research directions to design machine learning based components that contribute towards i-ZTA for the future 5G/6G networks.

arXiv:2105.00781공식 코드 · Papers with Code

Weakly supervised deep learning-based intracranial hemorrhage localization

Intracranial hemorrhage is a life-threatening disease, which requires fast medical intervention. Owing to the duration of data annotation, head CT images are usually available only with slice-level labeling. This paper presents a weakly supervised method of precise hemorrhage localization in axial slices using only position-free labels, which is based on multiple instance learning. An algorithm is introduced that generates hemorrhage likelihood maps and finds the coordinates of bleeding. The Dice coefficient of 58.08 % is achieved on data from a publicly available dataset.

arXiv:2104.14965

Unsupervised data augmentation for object detection

Data augmentation has always been an effective way to overcome overfitting issue when the dataset is small. There are already lots of augmentation operations such as horizontal flip, random crop or even Mixup. However, unlike image classification task, we cannot simply perform these operations for object detection task because of the lack of labeled bounding boxes information for corresponding generated images. To address this challenge, we propose a framework making use of Generative Adversarial Networks(GAN) to perform unsupervised data augmentation. To be specific, based on the recently supreme performance of YOLOv4, we propose a two-step pipeline that enables us to generate an image where the object lies in a certain position. In this way, we can accomplish the goal that generating an image with bounding box label.

arXiv:2104.14579공식 코드 · Papers with Code

LIDAR and Position-Aided mmWave Beam Selection with Non-local CNNs and Curriculum Training

Efficient millimeter wave (mmWave) beam selection in vehicle-to-infrastructure (V2I) communication is a crucial yet challenging task due to the narrow mmWave beamwidth and high user mobility. To reduce the search overhead of iterative beam discovery procedures, contextual information from light detection and ranging (LIDAR) sensors mounted on vehicles has been leveraged by data-driven methods to produce useful side information. In this paper, we propose a lightweight neural network (NN) architecture along with the corresponding LIDAR preprocessing, which significantly outperforms previous works. Our solution comprises multiple novelties that improve both the convergence speed and the final accuracy of the model. In particular, we define a novel loss function inspired by the knowledge distillation idea, introduce a curriculum training approach exploiting line-of-sight (LOS)/non-line-of-sight (NLOS) information, and we propose a non-local attention module to improve the performance for the more challenging NLOS cases. Simulation results on benchmark datasets show that, utilizing solely LIDAR data and the receiver position, our NN-based beam selection scheme can achieve 79.9% throughput of an exhaustive beam sweeping approach without any beam search overhead and 95% by searching among as few as 6 beams. In a typical mmWave V2I scenario, our proposed method considerably reduces the beam search time required to achieve a desired throughput, in comparison with the inverse fingerprinting and hierarchical beam selection schemes.

arXiv:2104.14075

UAV Swarm Position Optimization for High Capacity MIMO Backhaul

A swarm of cooperating UAVs communicating with a distant multiantenna ground station can leverage MIMO spatial multiplexing to scale the capacity. Due to the line-of-sight propagation between the swarm and the ground station, the MIMO channel is highly correlated, leading to limited multiplexing gains. In this paper, we optimize the UAV positions to attain the maximum MIMO capacity given by the single user bound. An infinite set of UAV placements that attains the capacity bound is first derived. Given an initial swarm placement, we formulate the problem of minimizing the distance traveled by the UAVs to reach a placement within the capacity maximizing set of positions. An offline centralized solution to the problem using block coordinate descent is developed assuming known initial positions of UAVs. We also propose an online distributed algorithm, where the UAVs iteratively adjust their positions to maximize the capacity. Our proposed approaches are shown to significantly increase the capacity at the expense of a bounded translation from the initial UAV placements. This capacity increase persists when using a massive MIMO ground station. Using numerical simulations, we show the robustness of our approaches in a Rician channel under UAV motion disturbances.

arXiv:2104.14528

GasHis-Transformer: A Multi-scale Visual Transformer Approach for Gastric Histopathological Image Detection

In this paper, a multi-scale visual transformer model, referred as GasHis-Transformer, is proposed for Gastric Histopathological Image Detection (GHID), which enables the automatic global detection of gastric cancer images. GasHis-Transformer model consists of two key modules designed to extract global and local information using a position-encoded transformer model and a convolutional neural network with local convolution, respectively. A publicly available hematoxylin and eosin (H&E) stained gastric histopathological image dataset is used in the experiment. Furthermore, a Dropconnect based lightweight network is proposed to reduce the model size and training time of GasHis-Transformer for clinical applications with improved confidence. Moreover, a series of contrast and extended experiments verify the robustness, extensibility and stability of GasHis-Transformer. In conclusion, GasHis-Transformer demonstrates high global detection performance and shows its significant potential in GHID task.

arXiv:2104.13044

Dual Transformer for Point Cloud Analysis

Following the tremendous success of transformer in natural language processing and image understanding tasks, in this paper, we present a novel point cloud representation learning architecture, named Dual Transformer Network (DTNet), which mainly consists of Dual Point Cloud Transformer (DPCT) module. Specifically, by aggregating the well-designed point-wise and channel-wise multi-head self-attention models simultaneously, DPCT module can capture much richer contextual dependencies semantically from the perspective of position and channel. With the DPCT module as a fundamental component, we construct the DTNet for performing point cloud analysis in an end-to-end manner. Extensive quantitative and qualitative experiments on publicly available benchmarks demonstrate the effectiveness of our proposed transformer framework for the tasks of 3D point cloud classification and segmentation, achieving highly competitive performance in comparison with the state-of-the-art approaches.

arXiv:2104.11124

Power Efficient Communication for Low Signal to Noise Ratio Optical Links

Receiver sensitivity is a particularly important metric in optical communication links operating at low signal-to-noise ratios (SNRs), for example in deep-space communication, since it directly limits the maximum achievable reach and data rate. Pulse position modulation (PPM) with direct detection photon-counting detectors are the most power-efficient solution known, however, the sensitivity gain comes at the expense of reduced spectral efficiency. We show that quadrature phase-shift keying (QPSK) modulation with a phase-sensitive ultralow noise pre-amplified coherent receiver outperforms other well-known power-efficient multi-dimensional coherent modulation formats, while simultaneously having higher spectral efficiency. It also results in better sensitivity than PPM for orders up to 64 with ideal direct detection using photon-counting receivers. This is because of the bit error rate characteristics favoring the QPSK format when forward error correction with a large overhead is considered.

arXiv:2104.09920

GPS-denied Navigation: Attitude, Position, Linear Velocity, and Gravity Estimation with Nonlinear Stochastic Observer

Successful navigation of a rigid-body traveling with six degrees of freedom (6 DoF) requires accurate estimation of attitude , position, and linear velocity. The true navigation dynamics are highly nonlinear and are modeled on the matrix Lie group of SE2(3). This paper presents novel geometric nonlinear continuous stochastic navigation observers on SE2(3) capturing the true nonlinearity of the problem. The proposed observers combines IMU and landmark measurements. It efficiently handles the IMU measurement noise. The proposed observers are guaranteed to be almost semi-globally uniformly ultimately bounded in the mean square. Quaternion representation is provided. A real-world quadrotor measurement dataset is used to validate the effectiveness of the proposed observers in its discrete form. Keywords: Inertial navigation, stochastic system, Brownian motion process, stochastic filter algorithm, stochastic differential equation, Lie group, SE(3), SO(3), pose estimator, position, attitude, feature measurement, inertial measurement unit, IMU.

arXiv:2104.09712

Problems and Countermeasures in Natural Language Processing Evaluation

Evaluation in natural language processing guides and promotes research on models and methods. In recent years, new evalua-tion data sets and evaluation tasks have been continuously proposed. At the same time, a series of problems exposed by ex-isting evaluation have also restricted the progress of natural language processing technology. Starting from the concept, com-position, development and meaning of natural language evaluation, this article classifies and summarizes the tasks and char-acteristics of mainstream natural language evaluation, and then summarizes the problems and causes of natural language pro-cessing evaluation. Finally, this article refers to the human language ability evaluation standard, puts forward the concept of human-like machine language ability evaluation, and proposes a series of basic principles and implementation ideas for hu-man-like machine language ability evaluation from the three aspects of reliability, difficulty and validity.

arXiv:2104.10157공식 코드 · Papers with Code

VideoGPT: Video Generation using VQ-VAE and Transformers

We present VideoGPT: a conceptually simple architecture for scaling likelihood based generative modeling to natural videos. VideoGPT uses VQ-VAE that learns downsampled discrete latent representations of a raw video by employing 3D convolutions and axial self-attention. A simple GPT-like architecture is then used to autoregressively model the discrete latents using spatio-temporal position encodings. Despite the simplicity in formulation and ease of training, our architecture is able to generate samples competitive with state-of-the-art GAN models for video generation on the BAIR Robot dataset, and generate high fidelity natural videos from UCF-101 and Tumbler GIF Dataset (TGIF). We hope our proposed architecture serves as a reproducible reference for a minimalistic implementation of transformer based video generation models. Samples and code are available at https://wilson1yan.github.io/videogpt/index.html

arXiv:2104.09981

Prospective Artificial Intelligence Approaches for Active Cyber Defence

Cybercriminals are rapidly developing new malicious tools that leverage artificial intelligence (AI) to enable new classes of adaptive and stealthy attacks. New defensive methods need to be developed to counter these threats. Some cybersecurity professionals are speculating AI will enable corresponding new classes of active cyber defence measures -- is this realistic, or currently mostly hype? The Alan Turing Institute, with expert guidance from the UK National Cyber Security Centre and Defence Science Technology Laboratory, published a research roadmap for AI for ACD last year. This position paper updates the roadmap for two of the most promising AI approaches -- reinforcement learning and causal inference - and describes why they could help tip the balance back towards defenders.

arXiv:2104.09864공식 코드 · Papers with Code

RoFormer: Enhanced Transformer with Rotary Position Embedding

Position encoding recently has shown effective in the transformer architecture. It enables valuable supervision for dependency modeling between elements at different positions of the sequence. In this paper, we first investigate various methods to integrate positional information into the learning process of transformer-based language models. Then, we propose a novel method named Rotary Position Embedding(RoPE) to effectively leverage the positional information. Specifically, the proposed RoPE encodes the absolute position with a rotation matrix and meanwhile incorporates the explicit relative position dependency in self-attention formulation. Notably, RoPE enables valuable properties, including the flexibility of sequence length, decaying inter-token dependency with increasing relative distances, and the capability of equipping the linear self-attention with relative position encoding. Finally, we evaluate the enhanced transformer with rotary position embedding, also called RoFormer, on various long text classification benchmark datasets. Our experiments show that it consistently overcomes its alternatives. Furthermore, we provide a theoretical analysis to explain some experimental results. RoFormer is already integrated into Huggingface: \url{https://huggingface.co/docs/transformers/model_doc/roformer}.

arXiv:2104.09612

LEx: A Framework for Operationalising Layers of Machine Learning Explanations

Several social factors impact how people respond to AI explanations used to justify AI decisions affecting them personally. In this position paper, we define a framework called the \textit{layers of explanation} (LEx), a lens through which we can assess the appropriateness of different types of explanations. The framework uses the notions of \textit{sensitivity} (emotional responsiveness) of features and the level of \textit{stakes} (decision's consequence) in a domain to determine whether different types of explanations are \textit{appropriate} in a given context. We demonstrate how to use the framework to assess the appropriateness of different types of explanations in different domains.

arXiv:2104.08698

A Simple and Effective Positional Encoding for Transformers

Transformer models are permutation equivariant. To supply the order and type information of the input tokens, position and segment embeddings are usually added to the input. Recent works proposed variations of positional encodings with relative position encodings achieving better performance. Our analysis shows that the gain actually comes from moving positional information to attention layer from the input. Motivated by this, we introduce Decoupled Positional Attention for Transformers (DIET), a simple yet effective mechanism to encode position and segment information into the Transformer models. The proposed method has faster training and inference time, while achieving competitive performance on GLUE, XTREME and WMT benchmarks. We further generalize our method to long-range transformers and show performance gain.

arXiv:2104.07538

Street-Map Based Validation of Semantic Segmentation in Autonomous Driving

Artificial intelligence for autonomous driving must meet strict requirements on safety and robustness, which motivates the thorough validation of learned models. However, current validation approaches mostly require ground truth data and are thus both cost-intensive and limited in their applicability. We propose to overcome these limitations by a model agnostic validation using a-priori knowledge from street maps. In particular, we show how to validate semantic segmentation masks and demonstrate the potential of our approach using OpenStreetMap. We introduce validation metrics that indicate false positive or negative road segments. Besides the validation approach, we present a method to correct the vehicle's GPS position so that a more accurate localization can be used for the street-map based validation. Lastly, we present quantitative results on the Cityscapes dataset indicating that our validation approach can indeed uncover errors in semantic segmentation masks.

arXiv:2012.06142공식 코드 · Papers with Code

Iterative Geometry Calibration from Distance Estimates for Wireless Acoustic Sensor Networks

In this paper we present an approach to geometry calibration in wireless acoustic sensor networks, whose nodes are assumed to be equipped with a compact microphone array. The proposed approach solely works with estimates of the distances between acoustic sources and the nodes that record these sources. It consists of an iterative weighted least squares localization procedure, which is initialized by multidimensional scaling. Alongside the sensor node locations, also the positions of the acoustic sources are estimated. Furthermore, we derive the Cramer-Rao lower bound (CRLB) for source and sensor position estimation, and show by simulation that the estimator is efficient.

arXiv:2005.02728

Deep Autoencoders for DOA Estimation of Coherent Sources using Imperfect Antenna Array

In this paper a robust algorithm for DOA estimation of coherent sources in presence of antenna array imperfections is presented. We exploit the current advances of deep learning to overcome two of the most common problems facing the state of the art DOA algorithms (i.e. coherent sources and array imperfections). We propose a deep auto encoder (AE) that is able to correctly resolve coherent sources without the need of spatial smoothing, hence avoiding possible processing overhead and delays. Moreover, we assumed the presence of array imperfections in the received signal model such as mutual coupling, gain/ phase mismatches, and position errors. The deep AE is trained using the covariance matrix of the received signal, where it alleviates the effect of imperfections, and at the same time act as a filters for the coherent sources. The results show significant improvement compared to the methods used in the literature.

arXiv:2005.00704

Automotive-Radar-Based 50-cm Urban Positioning

Deployment of automated ground vehicles (AGVs) beyond the confines of sunny and dry climes will require sub-lane-level positioning techniques based on radio waves rather than near-visible-light radiation. Like human sight, lidar and cameras perform poorly in low-visibility conditions. This paper develops and demonstrates a novel technique for robust 50-cm-accurate urban ground positioning based on commercially-available low-cost automotive radars. The technique is computationally efficient yet obtains a globally-optimal translation and heading solution, avoiding local minima caused by repeating patterns in the urban radar environment. Performance is evaluated on an extensive and realistic urban data set. Comparison against ground truth shows that, when coupled with stable short-term odometry, the technique maintains 95-percentile errors below 50 cm in horizontal position and 1 degree in heading.