There are hundreds of millions of tables in Web pages that contain useful information for many applications. Leveraging data within these tables is difficult because of the wide variety of structures, formats and data encoded in these tables. TabVec is an unsupervised method to embed tables into a vector space to support classification of tables into categories (entity, relational, matrix, list, and non-data) with minimal user intervention. TabVec deploys syntax and semantics of table cells, and embeds the structure of tables in a table vector space. This enables superior classification of tables even in the absence of domain annotations. Our evaluations in four real world domains show that TabVec improves classification accuracy by more than 20% compared to three state of the art systems, and that those systems require significant in domain training to achieve good results.
In this paper, we investigate the performance and success rates of Na\"ive Bayes Classification Algorithm for automatic classification of Turkish news into predetermined categories like economy, life, health etc. We use Apache Big Data technologies such as Hadoop, HDFS, Spark and Mahout, and apply these distributed technologies to Machine Learning.
Scene classification is a fundamental problem to understand the high-resolution remote sensing imagery. Recently, convolutional neural network (ConvNet) has achieved remarkable performance in different tasks, and significant efforts have been made to develop various representations for satellite image scene classification. In this paper, we present a novel representation based on a ConvNet with context aggregation. The proposed two-pathway ResNet (ResNet-TP) architecture adopts the ResNet as backbone, and the two pathways allow the network to model both local details and regional context. The ResNet-TP based representation is generated by global average pooling on the last convolutional layers from both pathways. Experiments on two scene classification datasets, UCM Land Use and NWPU-RESISC45, show that the proposed mechanism achieves promising improvements over state-of-the-art methods.
EEG is a non-invasive technique for recording brain bioelectric activity, which has potential applications in various fields such as human-computer interaction and neuroscience. However, there are many difficulties in analyzing EEG data, including its complex composition, low amplitude as well as low signal-to-noise ratio. Some of the existing methods of analysis are based on feature extraction and machine learning to differentiate the phase of schizophrenia that samples belong to. However, medical research requires the use of machine learning not only to give more accurate classification results, but also to give the results that can be applied to pathological studies. The main purpose of this study is to obtain the weight values as the representation of influence of each frequency band on the classification of schizophrenia phases on the basis of a more effective classification method using the LES feature extraction, and then the weight values are processed and applied to improve the accuracy of machine learning classification. We propose a method called weight-voting to obtain the weights of sub-bands features by using results of classification for voting to fit the actual categories of EEG data, and using weights for reclassification. Through this method, we can first obtain the influence of each band in distinguishing three schizophrenia phases, and analyze the effect of band features on the risk of schizophrenia contributing to the study of psychopathology. Our results show that there is a high correlation between the change of weight of low gamma band and the difference between HC, CHR and FES. If the features revised according to weights are used for reclassification, the accuracy of result will be improved compared with the original classifier, which confirms the role of the band weight distribution.
Resting state electroencephalogram (EEG) abnormalities in clinically high-risk individuals (CHR), clinically stable first-episode patients with schizophrenia (FES), healthy controls (HC) suggest alterations in neural oscillatory activity. However, few studies directly compare these anomalies among each types. Therefore, this study investigated whether these electrophysiological characteristics differentiate clinical populations from one another, and from non-psychiatric controls. To address this question, resting EEG power and coherence were assessed in 40 clinically high-risk individuals (CHR), 40 first-episode patients with schizophrenia (FES), and 40 healthy controls (HC). These findings suggest that resting EEG can be a sensitive measure for differentiating between clinical disorders.This paper proposes a novel data-driven supervised learning method to obtain identification of the patients mental status in schizophrenia research. According to Marchenko-Pastur Law, the distribution of the eigenvalues of EEG data is divided into signal subspace and noise subspace. A test statistic named LES that embodies the characteristics of all eigenvalues is adopted. different classifier and different feature(LES test function) are selected for experiments, we have shown that using von Neumann Entropy as LES test function combine with SVM classifier could obtain the best average classification accuracy during three classification among HC, FES and CHR of Schizophrenia group with EEG signal. It is worth noting that the result of LES feature extraction with the highest classification accuracy is around 90% in two classification(HC compare with FES) and around 70% in three classification. Where the classification accuracy higher than 70% could be used to assist clinical diagnosis.
A control logic has a central role in many echo cancellation systems for optimizing the performance of adaptive filters while estimating the echo path. For reliable control, accurate double-talk (DT) and channel change (CC) detectors are usually incorporated to the echo canceler. This work expands the usual detection strategy to define a classification problem characterizing four possible states of the echo canceler operation. The new formulation allow the use of decision theory to continuously control the transitions among the different modes of operation. The classification rule reduces to a low cost statistics for which it is possible to determine the probability of error under all hypotheses, allowing the classification performance to be accessed analytically. Monte Carlo simulations using synthetic and real data illustrate the reliability of the proposed method.
Web spam is a big problem for search engine users in World Wide Web. They use deceptive techniques to achieve high rankings. Although many researchers have presented the different approach for classification and web spam detection still it is an open issue in computer science. Analyzing and evaluating these websites can be an effective step for discovering and categorizing the features of these websites. There are several methods and algorithms for detecting those websites, such as decision tree algorithm. In this paper, we present a systematic framework based on CHAID algorithm and a modified string matching algorithm (KMP) for extract features and analysis of these websites. We evaluated our model and other methods with a dataset of Alexa Top 500 Global Sites and Bing search engine results in 500 queries.
We consider cell line classification using multivariate time series data obtained from electric cell-substrate impedance sensing (ECIS) technology. The ECIS device, which monitors the attachment and spreading of mammalian cells in real time through the collection of electrical impedance data, has historically been used to study one cell line at a time. However, we show that if applied to data from multiple cell lines, ECIS can be used to classify unknown or potentially mislabeled cells, which may help to mitigate the current crisis of reproducibility in the biological literature. We assess a range of approaches to this new problem, testing different classification methods and deriving a dictionary of 29 features to characterize ECIS data. Our analysis also makes use of simultaneous multi-frequency ECIS data, where previous studies have focused on only one frequency. In classification tests on fifteen mammalian cell lines, we obtain very high out-of-sample accuracy. These preliminary findings provide a baseline for future large-scale studies in this field.
This short empirical paper investigates how well topic modeling and database meta-data characteristics can classify web and other proof-of-concept (PoC) exploits for publicly disclosed software vulnerabilities. By using a dataset comprised of over 36 thousand PoC exploits, near a 0.9 accuracy rate is obtained in the empirical experiment. Text mining and topic modeling are a significant boost factor behind this classification performance. In addition to these empirical results, the paper contributes to the research tradition of enhancing software vulnerability information with text mining, providing also a few scholarly observations about the potential for semi-automatic classification of exploits in the existing tracking infrastructures.
The name disambiguation task partitions a collection of records pertaining to a given name, such that there is a one-to-one correspondence between the partitions and a group of people, all sharing that given name. Most existing solutions for this task are proposed for static data. However, more realistic scenarios stipulate emergence of records in a streaming fashion where records may belong to known as well as unknown persons all sharing the same name. This requires a flexible name disambiguation algorithm that can not only classify records of known persons represented in the train- ing data by their existing records but can also identify records of new ambiguous persons with no existing records included in the initial training dataset. Toward achieving this objective, in this paper we propose a Bayesian non-exhaustive classification frame- work for solving online name disambiguation. In particular, we present a Dirichlet Process Gaussian Mixture Model (DPGMM) as a core engine for online name disambiguation task. Meanwhile, two online inference algorithms, namely one-pass Gibbs sampler and Sequential Importance Sampling with Resampling (also known as particle filtering), are proposed to simultaneously perform online classification and new class discovery. As a case study we consider bibliographic data in a temporal stream format and disambiguate authors by partitioning their papers into homogeneous groups.Our experimental results demonstrate that the proposed method is significantly better than existing methods for performing online name disambiguation task. We also propose an interactive version of our online name disambiguation method designed to leverage user feedback to improve prediction accuracy.
The purpose of this paper is to focus on similarity and/or heterogeneity of taxonomies of innovation present in the economic fields to show as the economic literature uses different names to indicate the same type of technical change and innovation, and the same name for different types of innovation. This ambiguity of classification makes it impossible to compare the various studies; moreover the numerous typologies existing in the economics of innovation, technometrics, economics of technical change, management of technology, etc., have hindered the development of knowledge in these fields. The research presents also new directions on the classification of innovation that try to overcome these problems.
The vision of the Semantic Web (SW) is gradually unfolding and taking shape through a web of linked data, a part of which is built by capturing semantics stored in existing knowledge organization systems (KOS), subject metadata and resource metadata. The content of vast bibliographic collections is currently categorized by some widely used bibliographic classification and we may soon see them being mined for information and linked in a meaningful way across the Web. Bibliographic classifications are designed for knowledge mediation which offers both a rich terminology and different ways in which concepts can be categorized and related to each other in the universe of knowledge. From 1990-2010 they have been used in various resource discovery services on the Web and continue to be used to support information integration in a number of international digital library projects. In this chapter we will revisit some of the ways in which universal classifications, as language independent concept schemes, can assist humans and computers in structuring and presenting information and formulating queries. Most importantly, we highlight issues important to understanding bibliographic classifications, both in terms of their unused potential and technical limitations.
The paper discusses issues related to the use of faceted classifications in an online environment. The author argues that knowledge organization systems can be fully utilized in information retrieval only if they are exposed and made available for machine processing. The experience with classification automation to date may be used to speed up and ease the conversion of existing faceted schemes or the creation of management tools for new systems. The author suggests that it is possible to agree on a set of functional requirements for supporting faceted classifications online that are equally relevant for the maintenance of classifications, the creation of classification indexing tools, or the management of classifications in an authority file. It is suggested that a set of requirements for analytico-synthetic classifications may be put forward to improve standards for the use and exchange of knowledge organization systems.
The absence of an appropriate text classification corpus makes the massive amount of online job information unusable for labor market analysis. This paper presents JCTC, a large job posting corpus for text classification. In JCTC construction framework, a formal specification issued by the Chinese central government is chosen as the classification standard. The unsupervised learning (WE-cos), supervised learning algorithm (SVM) and human judgements are all used in the construction process. JCTC has 102581 online job postings distributed in 465 categories. The method proposed here can not only ameliorate the high demands on people's skill and knowledge, but reduce the subjective influences as well. Besides, the method is not limited in Chinese. We benchmark five state-of-the-art deep learning approaches on JCTC providing baseline results for future studies. JCTC might be the first job posting corpus for text classification and the largest one in Chinese. With the help of JCTC, related organizations are able to monitor, analyze and predict the labor market in a comprehensive, accurate and timely manner.
Although brain circuits presumably carry out useful perceptual algorithms, few instances of derived biological methods have been found to compete favorably against algorithms that have been engineered for specific applications. We forward a novel analysis of function of cortico-striatal loops, which constitute more than 80% of the human brain, thus likely underlying a broad range of cognitive functions. We describe a family of operations performed by the derived method, including a nonstandard method for supervised classification, which may underlie some forms of cortically-dependent associative learning. The novel supervised classifier is compared against widely-used algorithms for classification, including support vector machines (SVM) and k-nearest neighbor methods, achieving corresponding classification rates --- at a fraction of the time and space costs. This represents an instance of a biologically-derived algorithm comparing favorably against widely used machine learning methods on well-studied tasks.
The correct prediction of protein secondary structures is one of the key issues in predicting the correct protein folded shape, which is used for determining gene function. Existing methods make use of amino acids properties as indices to classify protein secondary structures, but are faced with a significant number of misclassifications. The paper presents a technique for the classification of protein secondary structures based on protein "signal-plotting" and the use of the Fourier technique for digital signal processing. New indices are proposed to classify protein secondary structures by analyzing hydrophobicity profiles. The approach is simple and straightforward. Results show that the more types of protein secondary structures can be classified by means of these newly-proposed indices.
We tackle the challenge of topic classification of tweets in the context of analyzing a large collection of curated streams by news outlets and other organizations to deliver relevant content to users. Our approach is novel in applying distant supervision based on semi-automatically identifying curated streams that are topically focused (for example, on politics, entertainment, or sports). These streams provide a source of labeled data to train topic classifiers that can then be applied to categorize tweets from more topically-diffuse streams. Experiments on both noisy labels and human ground-truth judgments demonstrate that our approach yields good topic classifiers essentially "for free", and that topic classifiers trained in this manner are able to dynamically adjust for topic drift as news on Twitter evolves.
Almost by definition, radical innovations create a need to revise existing classification systems. In this paper, we argue that classification system changes and patent reclassification are common and reveal interesting information about technological evolution. To support our argument, we present three sets of findings regarding classification volatility in the U.S. patent classification system. First, we study the evolution of the number of distinct classes. Reconstructed time series based on the current classification scheme are very different from historical data. This suggests that using the current classification to analyze the past produces a distorted view of the evolution of the system. Second, we study the relative sizes of classes. The size distribution is exponential so classes are of quite different sizes, but the largest classes are not necessarily the oldest. To explain this pattern with a simple stochastic growth model, we introduce the assumption that classes have a regular chance to be split. Third, we study reclassification. The share of patents that are in a different class now than they were at birth can be quite high. Reclassification mostly occurs across classes belonging to the same 1-digit NBER category, but not always. We also document that reclassified patents tend to be more cited than non-reclassified ones, even after controlling for grant year and class of origin.
The impact of social media and its growing association with the sharing of ideas and propagation of messages remains vital in everyday communication. Twitter is one effective platform for the dissemination of news and stories about recent events happening around the world. It has a continually growing database currently adopted by over 300 million users. In this paper we propose a novel grid-based approach employing supervised Multinomial Naive Bayes while extracting geographic entities from relevant user descriptions metadata which gives a spatial indication of the user location. To the best of our knowledge our approach is the first to make location inference from tweets using geo-enriched grid-based classification. Our approach performs better than existing baselines achieving more than 57% accuracy at city-level granularity. In addition we present a novel framework for content-based estimation of user locations by specifying levels of granularity required in pre-defined location grids.
We give complete algorithms and source code for constructing (multilevel) statistical industry classifications, including methods for fixing the number of clusters at each level (and the number of levels). Under the hood there are clustering algorithms (e.g., k-means). However, what should we cluster? Correlations? Returns? The answer turns out to be neither and our backtests suggest that these details make a sizable difference. We also give an algorithm and source code for building "hybrid" industry classifications by improving off-the-shelf "fundamental" industry classifications by applying our statistical industry classification methods to them. The presentation is intended to be pedagogical and geared toward practical applications in quantitative trading.