PyTorchKR Papers

최신 논문

총 11,126편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2108.02283

Machine Learning Classification Methods and Portfolio Allocation: An Examination of Market Efficiency

We design a novel framework to examine market efficiency through out-of-sample (OOS) predictability. We frame the asset pricing problem as a machine learning classification problem and construct classification models to predict return states. The prediction-based portfolios beat the market with significant OOS economic gains. We measure prediction accuracies directly. For each model, we introduce a novel application of binomial test to test the accuracy of 3.34 million return state predictions. The tests show that our models can extract useful contents from historical information to predict future return states. We provide unique economic insights about OOS predictability and machine learning models.

arXiv:2108.02432공식 코드 · Papers with Code

Token Shift Transformer for Video Classification

Transformer achieves remarkable successes in understanding 1 and 2-dimensional signals (e.g., NLP and Image Content Understanding). As a potential alternative to convolutional neural networks, it shares merits of strong interpretability, high discriminative power on hyper-scale data, and flexibility in processing varying length inputs. However, its encoders naturally contain computational intensive operations such as pair-wise self-attention, incurring heavy computational burden when being applied on the complex 3-dimensional video signals. This paper presents Token Shift Module (i.e., TokShift), a novel, zero-parameter, zero-FLOPs operator, for modeling temporal relations within each transformer encoder. Specifically, the TokShift barely temporally shifts partial [Class] token features back-and-forth across adjacent frames. Then, we densely plug the module into each encoder of a plain 2D vision transformer for learning 3D video representation. It is worth noticing that our TokShift transformer is a pure convolutional-free video transformer pilot with computational efficiency for video understanding. Experiments on standard benchmarks verify its robustness, effectiveness, and efficiency. Particularly, with input clips of 8/12 frames, the TokShift transformer achieves SOTA precision: 79.83%/80.40% on the Kinetics-400, 66.56% on EGTEA-Gaze+, and 96.80% on UCF-101 datasets, comparable or better than existing SOTA convolutional counterparts. Our code is open-sourced in: https://github.com/VideoNetworks/TokShift-Transformer.

arXiv:2108.02428

A Method for Medical Data Analysis Using the LogNNet for Clinical Decision Support Systems and Edge Computing in Healthcare

Edge computing is a fast-growing and much needed technology in healthcare. The problem of implementing artificial intelligence on edge devices is the complexity and high resource intensity of the most known neural network data analysis methods and algorithms. The difficulty of implementing these methods on low-power microcontrollers with small memory size calls for the development of new effective algorithms for neural networks. This study presents a new method for analyzing medical data based on the LogNNet neural network, which uses chaotic mappings to transform input information. The method effectively solves classification problems and calculates risk factors for the presence of a disease in a patient according to a set of medical health indicators. The efficiency of LogNNet in assessing perinatal risk is illustrated on cardiotocogram data obtained from the UC Irvine machine learning repository. The classification accuracy reaches ~91% with the ~3-10 kB of RAM used on the Arduino microcontroller. Using the LogNNet network trained on a publicly available database of the Israeli Ministry of Health, a service concept for COVID-19 express testing is provided. A classification accuracy of ~95% is achieved, and ~0.6 kB of RAM is used. In all examples, the model is tested using standard classification quality metrics: precision, recall, and F1-measure. The LogNNet architecture allows the implementation of artificial intelligence on medical peripherals of the Internet of Things with low RAM resources and can be used in clinical decision support systems.

arXiv:2108.02241

Attentive Cross-modal Connections for Deep Multimodal Wearable-based Emotion Recognition

Classification of human emotions can play an essential role in the design and improvement of human-machine systems. While individual biological signals such as Electrocardiogram (ECG) and Electrodermal Activity (EDA) have been widely used for emotion recognition with machine learning methods, multimodal approaches generally fuse extracted features or final classification/regression results to boost performance. To enhance multimodal learning, we present a novel attentive cross-modal connection to share information between convolutional neural networks responsible for learning individual modalities. Specifically, these connections improve emotion classification by sharing intermediate representations among EDA and ECG and apply attention weights to the shared information, thus learning more effective multimodal embeddings. We perform experiments on the WESAD dataset to identify the best configuration of the proposed method for emotion classification. Our experiments show that the proposed approach is capable of learning strong multimodal representations and outperforms a number of baselines methods.

arXiv:2108.02035공식 코드 · Papers with Code

Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification

Tuning pre-trained language models (PLMs) with task-specific prompts has been a promising approach for text classification. Particularly, previous studies suggest that prompt-tuning has remarkable superiority in the low-data scenario over the generic fine-tuning methods with extra classifiers. The core idea of prompt-tuning is to insert text pieces, i.e., template, to the input and transform a classification problem into a masked language modeling problem, where a crucial step is to construct a projection, i.e., verbalizer, between a label space and a label word space. A verbalizer is usually handcrafted or searched by gradient descent, which may lack coverage and bring considerable bias and high variances to the results. In this work, we focus on incorporating external knowledge into the verbalizer, forming a knowledgeable prompt-tuning (KPT), to improve and stabilize prompt-tuning. Specifically, we expand the label word space of the verbalizer using external knowledge bases (KBs) and refine the expanded label word space with the PLM itself before predicting with the expanded label word space. Extensive experiments on zero and few-shot text classification tasks demonstrate the effectiveness of knowledgeable prompt-tuning.

arXiv:2108.01921

TextCNN with Attention for Text Classification

The vast majority of textual content is unstructured, making automated classification an important task for many applications. The goal of text classification is to automatically classify text documents into one or more predefined categories. Recently proposed simple architectures for text classification such as Convolutional Neural Networks for Sentence Classification by Kim, Yoon showed promising results. In this paper, we propose incorporating an attention mechanism into the network to boost its performance, we also propose WordRank for vocabulary selection to reduce the network embedding parameters and speed up training with minimum accuracy loss. By adopting the proposed ideas TextCNN accuracy on 20News increased from 94.79 to 96.88, moreover, the number of parameters for the embedding layer can be reduced substantially with little accuracy loss by using WordRank. By using WordRank for vocabulary selection we can reduce the number of parameters by more than 5x from 7.9M to 1.5M, and the accuracy will only decrease by 1.2%.

arXiv:2108.01668

Classification of Electrical Impedance Tomography Data Using Machine Learning

Patients suffering from pulmonary diseases typically exhibit pathological lung ventilation in terms of homogeneity. Electrical Impedance Tomography (EIT) is a non-invasive imaging method that allows to analyze and quantify the distribution of ventilation in the lungs. In this article, we present a new approach to promote the use of EIT data and the implementation of new clinical applications for differential diagnosis, with the development of several machine learning models to discriminate between EIT data from healthy and non-healthy subjects. EIT data from 16 subjects were acquired: 5 healthy and 11 non-healthy subjects (with multiple pulmonary conditions). Preliminary results have shown accuracy percentages of 66\% in challenging evaluation scenarios. The results suggest that the pairing of EIT feature engineering methods with machine learning methods could be further explored and applied in the diagnostic and monitoring of patients suffering from lung diseases. Also, we introduce the use of a new feature in the context of EIT data analysis (Impedance Curve Correlation).

논문 ID: document/1453511

Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy

Feature selection is an important problem for pattern classification systems. We study how to select good features according to the maximal statistical dependency criterion based on mutual information. Because of the difficulty in directly implementing the maximal dependency condition, we first derive an equivalent form, called minimal-redundancy-maximal-relevance criterion (mRMR), for first-order incremental feature selection. Then, we present a two-stage feature selection algorithm by combining mRMR and other more sophisticated feature selectors (e.g., wrappers). This allows us to select a compact set of superior features at very low cost. We perform extensive experimental comparison of our algorithm and other methods using three different classifiers (naive Bayes, support vector machine, and linear discriminate analysis) and four different data sets (handwritten digits, arrhythmia, NCI cancer cell lines, and lymphoma tissues). The results confirm that mRMR leads to promising improvement on feature selection and classification accuracy.Feature selection is an important problem for pattern classification systems. We study how to select good features according to the maximal statistical dependency criterion based on mutual information. Because of the difficulty in directly implementing the maximal dependency condition, we first derive an equivalent form, called minimal-redundancy-maximal-relevance criterion (mRMR), for first-order incremental feature selection. Then, we present a two-stage feature selection algorithm by combining mRMR and other more sophisticated feature selectors (e.g., wrappers). This allows us to select a compact set of superior features at very low cost. We perform extensive experimental comparison of our algorithm and other methods using three different classifiers (naive Bayes, support vector machine, and linear discriminate analysis) and four different data sets (handwritten digits, arrhythmia, NCI cancer cell lines, and lymphoma tissues). The results confirm that mRMR leads to promising improvement on feature selection and classification accuracy.

arXiv:2108.01656

Open Set Wireless Standard Classification Using Convolutional Neural Networks

In congested electromagnetic environments, cognitive radios require knowledge about other emitters in order to optimize their dynamic spectrum access strategy. Deep learning classification algorithms have been used to recognize the wireless signal standards of emitters with high accuracy, but are limited to classifying signal classes that appear in their training set. This diminishes the performance of deep learning classifiers deployed in the field because they cannot accurately identify signals from classes outside of the training set. In this paper, a convolution neural network based open set classifier is proposed with the ability to detect if signals are not from known classes by thresholding the output sigmoid activation. The open set classifier was trained on 4G LTE, 5G NR, IEEE 802.11ax, Bluetooth Low Energy 5.0, and Narrowband Internet-of-Things signals impaired with Rayleigh or Rician fading, AWGN, frequency offsets, and in-phase/quadrature imbalances. Then, the classifier was tested on OFDM, SC-FDMA, SC, AM, and FM signals, which did not appear in the training set classes. The closed set classifier achieves an average accuracy of 94.5% for known signals with SNR's greater than 0 dB, but by design, has a 0% accuracy detecting signals from unknown classes. On the other hand, the open set classifier retains an 86% accuracy for known signal classes, but can detect 95.5% of signals from unknown classes with SNR's greater than 0 dB.

arXiv:2108.01640공식 코드 · Papers with Code

Automatic classification of eclipsing binary stars using deep learning methods

In the last couple of decades, tremendous progress has been achieved in developing robotic telescopes and, as a result, sky surveys (both terrestrial and space) have become the source of a substantial amount of new observational data. These data contain a lot of information about binary stars, hidden in their light curves. With the huge amount of astronomical data gathered, it is not reasonable to expect all the data to be manually processed and analyzed. Therefore, in this paper, we focus on the automatic classification of eclipsing binary stars using deep learning methods. Our classifier provides a tool for the categorization of light curves of binary stars into two classes: detached and over-contact. We used the ELISa software to obtain synthetic data, which we then used for the training of the classifier. For evaluation purposes, we collected 100 light curves of observed binary stars, in order to evaluate a number of classifiers. We evaluated semi-detached eclipsing binary stars as detached. The best-performing classifier combines bidirectional Long Short-Term Memory (LSTM) and a one-dimensional convolutional neural network, which achieved 98% accuracy on the evaluation set. Omitting semi-detached eclipsing binary stars, we could obtain 100% accuracy in classification.

arXiv:2108.01573

Classification of Discrete Dynamical Systems Based on Transients

In order to develop systems capable of artificial evolution, we need to identify which systems can produce complex behavior. We present a novel classification method applicable to any class of deterministic discrete space and time dynamical systems. The method is based on classifying the asymptotic behavior of the average computation time in a given system before entering a loop. We were able to identify a critical region of behavior that corresponds to a phase transition from ordered behavior to chaos across various classes of dynamical systems. To show that our approach can be applied to many different computational systems, we demonstrate the results of classifying cellular automata, Turing machines, and random Boolean networks. Further, we use this method to classify 2D cellular automata to automatically find those with interesting, complex dynamics. We believe that our work can be used to design systems in which complex structures emerge. Also, it can be used to compare various versions of existing attempts to model open-ended evolution (Ray (1991), Ofria et al. (2004), Channon (2006)).

arXiv:2108.01513

SphereFace2: Binary Classification is All You Need for Deep Face Recognition

State-of-the-art deep face recognition methods are mostly trained with a softmax-based multi-class classification framework. Despite being popular and effective, these methods still have a few shortcomings that limit empirical performance. In this paper, we start by identifying the discrepancy between training and evaluation in the existing multi-class classification framework and then discuss the potential limitations caused by the "competitive" nature of softmax normalization. Motivated by these limitations, we propose a novel binary classification training framework, termed SphereFace2. In contrast to existing methods, SphereFace2 circumvents the softmax normalization, as well as the corresponding closed-set assumption. This effectively bridges the gap between training and evaluation, enabling the representations to be improved individually by each binary classification task. Besides designing a specific well-performing loss function, we summarize a few general principles for this "one-vs-all" binary classification framework so that it can outperform current competitive methods. Our experiments on popular benchmarks demonstrate that SphereFace2 can consistently outperform state-of-the-art deep face recognition methods. The code has been made publicly available.

arXiv:2108.01394

AI Based Waste classifier with Thermo-Rapid Composting

Waste management is a certainly a very complex and difficult process especially in very large cities. It needs immense man power and also uses up other resources such as electricity and fuel. This creates a need to use a novel method with help of latest technologies. Here in this article we present a new waste classification technique using Computer Vision (CV) and deep learning (DL). To further improve waste classification ability, support machine vectors (SVM) are used. We also decompose the degradable waste with help of rapid composting. In this article we have mainly worked on segregation of municipal solid waste (MSW). For this model, we use YOLOv3 (You Only Look Once) a computer vision-based algorithm popularly used to detect objects which is developed based on Convolution Neural Networks (CNNs) which is a machine learning (ML) based tool. They are extensively used to extract features from a data especially image-oriented data. In this article we propose a waste classification technique which will be faster and more efficient. And we decompose the biodegradable waste by Berkley Method of composting (BKC)

arXiv:2108.00829

Information-theoretic symmetry classifications of crystal patterns in the presence of noise and strong Fedorov type pseudosymmetries for an optimal subsequent crystallographic processing of these patterns

Statistically sound crystallographic symmetry classifications are obtained with information theory based methods in the presence of approximately Gaussian distributed noise. A set of three synthetic images with very strong Fedorov type pseudosymmetries and varying amounts of noise serve as examples. The correct distinctions between genuine symmetries and their Fedorov type pseudosymmetry counterparts failed only for the noisiest image of the series where an inconsistent combination of plane symmetry group and projected Laue class was obtained. Contrary to traditional crystallographic symmetry classifications with an image processing program such as CRISP, the classification process does not need to be supervised by a human being. This enables crystallographic symmetry classification of digital images that are more or less periodic in two dimensions (2D) as recorded with sufficient spatial resolution from a wide range of samples with different types of scanning probe microscopes. Alternatives to the employed objective classification methods as proposed by members of the computational symmetry community and machine learning proponents are briefly discussed in an appendix and are found to be wanting because they ignore Fedorov type pseudosymmetries completely. The information theory based methods are more accurate than visual classifications at first sight by most human experts.

arXiv:2108.00261공식 코드 · Papers with Code

ECLARE: Extreme Classification with Label Graph Correlations

Deep extreme classification (XC) seeks to train deep architectures that can tag a data point with its most relevant subset of labels from an extremely large label set. The core utility of XC comes from predicting labels that are rarely seen during training. Such rare labels hold the key to personalized recommendations that can delight and surprise a user. However, the large number of rare labels and small amount of training data per rare label offer significant statistical and computational challenges. State-of-the-art deep XC methods attempt to remedy this by incorporating textual descriptions of labels but do not adequately address the problem. This paper presents ECLARE, a scalable deep learning architecture that incorporates not only label text, but also label correlations, to offer accurate real-time predictions within a few milliseconds. Core contributions of ECLARE include a frugal architecture and scalable techniques to train deep models along with label correlation graphs at the scale of millions of labels. In particular, ECLARE offers predictions that are 2 to 14% more accurate on both publicly available benchmark datasets as well as proprietary datasets for a related products recommendation task sourced from the Bing search engine. Code for ECLARE is available at https://github.com/Extreme-classification/ECLARE.

arXiv:2108.01019

A Framework for Multi-View Classification of Features

One of the most important problems in the field of pattern recognition is data classification. Due to the increasing development of technologies introduced in the field of data classification, some of the solutions are still open and need more research. One of the challenging problems in this area is the curse of dimensionality of the feature set of the data classification problem. In solving the data classification problems, when the feature set is too large, typical approaches will not be able to solve the problem. In this case, an approach can be used to partition the feature set into multiple feature sub-sets so that the data classification problem is solved for each of the feature subsets and finally using the ensemble classification, the classification is applied to the entire feature set. In the above-mentioned approach, the partitioning of feature set into feature sub-sets is still an interesting area in the literature of this field. In this research, an innovative framework for multi-view ensemble classification, inspired by the problem of object recognition in the multiple views theory of humans, is proposed. In this method, at first, the collaboration values between the features is calculated using a criterion called the features collaboration criterion. Then, the collaboration graph is formed based on the calculated collaboration values. In the next step, using the community detection method, graph communities are found. The communities are considered as the problem views and the different base classifiers are trained for different views using the views corresponding training data. The multi-view ensemble classifier is then formed by a combination of base classifiers based on the AdaBoost algorithm. The simulation results of the proposed method on the real and synthetic datasets show that the proposed method increases the classification accuracy.

arXiv:2108.00969

Convergence rates of deep ReLU networks for multiclass classification

For classification problems, trained deep neural networks return probabilities of class memberships. In this work we study convergence of the learned probabilities to the true conditional class probabilities. More specifically we consider sparse deep ReLU network reconstructions minimizing cross-entropy loss in the multiclass classification setup. Interesting phenomena occur when the class membership probabilities are close to zero. Convergence rates are derived that depend on the near-zero behaviour via a margin-type condition.

arXiv:2108.00813

The CirCor DigiScope Dataset: From Murmur Detection to Murmur Classification

Cardiac auscultation is one of the most cost-effective techniques used to detect and identify many heart conditions. Computer-assisted decision systems based on auscultation can support physicians in their decisions. Unfortunately, the application of such systems in clinical trials is still minimal since most of them only aim to detect the presence of extra or abnormal waves in the phonocardiogram signal, i.e., only a binary ground truth variable (normal vs abnormal) is provided. This is mainly due to the lack of large publicly available datasets, where a more detailed description of such abnormal waves (e.g., cardiac murmurs) exists. To pave the way to more effective research on healthcare recommendation systems based on auscultation, our team has prepared the currently largest pediatric heart sound dataset. A total of 5282 recordings have been collected from the four main auscultation locations of 1568 patients, in the process, 215780 heart sounds have been manually annotated. Furthermore, and for the first time, each cardiac murmur has been manually annotated by an expert annotator according to its timing, shape, pitch, grading, and quality. In addition, the auscultation locations where the murmur is present were identified as well as the auscultation location where the murmur is detected more intensively. Such detailed description for a relatively large number of heart sounds may pave the way for new machine learning algorithms with a real-world application for the detection and analysis of murmur waves for diagnostic purposes.

arXiv:2108.00368공식 코드 · Papers with Code

DECAF: Deep Extreme Classification with Label Features

Extreme multi-label classification (XML) involves tagging a data point with its most relevant subset of labels from an extremely large label set, with several applications such as product-to-product recommendation with millions of products. Although leading XML algorithms scale to millions of labels, they largely ignore label meta-data such as textual descriptions of the labels. On the other hand, classical techniques that can utilize label metadata via representation learning using deep networks struggle in extreme settings. This paper develops the DECAF algorithm that addresses these challenges by learning models enriched by label metadata that jointly learn model parameters and feature representations using deep networks and offer accurate classification at the scale of millions of labels. DECAF makes specific contributions to model architecture design, initialization, and training, enabling it to offer up to 2-6% more accurate prediction than leading extreme classifiers on publicly available benchmark product-to-product recommendation datasets, such as LF-AmazonTitles-1.3M. At the same time, DECAF was found to be up to 22x faster at inference than leading deep extreme classifiers, which makes it suitable for real-time applications that require predictions within a few milliseconds. The code for DECAF is available at the following URL https://github.com/Extreme-classification/DECAF.

arXiv:2108.00340

Reconstruction guided Meta-learning for Few Shot Open Set Recognition

In many applications, we are constrained to learn classifiers from very limited data (few-shot classification). The task becomes even more challenging if it is also required to identify samples from unknown categories (open-set classification). Learning a good abstraction for a class with very few samples is extremely difficult, especially under open-set settings. As a result, open-set recognition has received minimal attention in the few-shot setting. However, it is a critical task in many applications like environmental monitoring, where the number of labeled examples for each class is limited. Existing few-shot open-set recognition (FSOSR) methods rely on thresholding schemes, with some considering uniform probability for open-class samples. However, this approach is often inaccurate, especially for fine-grained categorization, and makes them highly sensitive to the choice of a threshold. To address these concerns, we propose Reconstructing Exemplar-based Few-shot Open-set ClaSsifier (ReFOCS). By using a novel exemplar reconstruction-based meta-learning strategy ReFOCS streamlines FSOSR eliminating the need for a carefully tuned threshold by learning to be self-aware of the openness of a sample. The exemplars, act as class representatives and can be either provided in the training dataset or estimated in the feature domain. By testing on a wide variety of datasets, we show ReFOCS to outperform multiple state-of-the-art methods.