Extracting effective features is always a challenging problem for texture classification because of the uncertainty of scales and the clutter of textural patterns. For texture classification, spectral analysis is traditionally employed in the frequency domain. Recent studies have shown the potential of convolutional neural networks (CNNs) when dealing with the texture classification task in the spatial domain. In this article, we try combining both approaches in different domains for more abundant information and proposed a novel network architecture named contourlet CNN (C-CNN). The network aims to learn sparse and effective feature representations for images. First, the contourlet transform is applied to get the spectral features from an image. Second, the spatial-spectral feature fusion strategy is designed to incorporate the spectral features into CNN architecture. Third, the statistical features are integrated into the network by the statistical feature fusion. Finally, the results are obtained by classifying the fusion features. We also investigated the behavior of the parameters in contourlet decomposition. Experiments on the widely used three texture data sets (kth-tips2-b, DTD, and CUReT) and five remote sensing data sets (UCM, WHU-RS, AID, RSSCN7, and NWPU-RESISC45) demonstrate that the proposed approach outperforms several well-known classification methods in terms of classification accuracy with fewer trainable parameters.
Lung cancer is a leading cause of death in most countries of the world. Since prompt diagnosis of tumors can allow oncologists to discern their nature, type and the mode of treatment, tumor detection and segmentation from CT Scan images is a crucial field of study worldwide. This paper approaches lung tumor segmentation by applying two-dimensional discrete wavelet transform (DWT) on the LOTUS dataset for more meticulous texture analysis whilst integrating information from neighboring CT slices before feeding them to a Deeply Supervised MultiResUNet model. Variations in learning rates, decay and optimization algorithms while training the network have led to different dice co-efficients, the detailed statistics of which have been included in this paper. We also discuss the challenges in this dataset and how we opted to overcome them. In essence, this study aims to maximize the success rate of predicting tumor regions from two dimensional CT Scan slices by experimenting with a number of adequate networks, resulting in a dice co-efficient of 0.8472.
Surface roughness plays an important role in analyzing engineering surfaces. It quantifies the surface topography and can be used to determine whether the resulting surface finish is acceptable or not. Nevertheless, while several existing tools and standards are available for computing surface roughness, these methods rely heavily on user input thus slowing down the analysis and increasing manufacturing costs. Therefore, fast and automatic determination of the roughness level is essential to avoid costs resulting from surfaces with unacceptable finish, and user-intensive analysis. In this study, we propose a Topological Data Analysis (TDA) based approach to classify the roughness level of synthetic surfaces using both their areal images and profiles. We utilize persistent homology from TDA to generate persistence diagrams that encapsulate information on the shape of the surface. We then obtain feature matrices for each surface or profile using Carlsson coordinates, persistence images, and template functions. We compare our results to two widely used methods in the literature: Fast Fourier Transform (FFT) and Gaussian filtering. The results show that our approach yields mean accuracies as high as 97%. We also show that, in contrast to existing surface analysis tools, our TDA-based approach is fully automatable and provides adaptive feature extraction.
Here we propose a new method for the classification of texture images combining fractal measures (fractal dimension, multifractal spectrum and lacunarity) with local binary patterns. More specifically we compute the box counting dimension of the local binary codes thresholded at different levels to compose the feature vector. The proposal is assessed in the classification of three benchmark databases: KTHTIPS-2b, UMD and UIUC as well as in a real-world problem, namely the identification of Brazilian plant species (database 1200Tex) using scanned images of their leaves. The proposed method demonstrated to be competitive with other state-of-the-art solutions reported in the literature. Such results confirmed the potential of combining a powerful local coding description with the multiscale information captured by the fractal dimension for texture classification.
Here we propose and investigate the use of visibility graphs to model the feature map of a neural network. The model, initially devised for studies on complex networks, is employed here for the classification of texture images. The work is motivated by an alternative viewpoint provided by these graphs over the original data. The performance of the proposed method is verified in the classification of four benchmark databases, namely, KTHTIPS-2b, FMD, UIUC, and UMD and in a practical problem, which is the identification of plant species using scanned images of their leaves. Our method was competitive with other state-of-the-art approaches, confirming the potential of techniques used for data analysis in different contexts to give more meaningful interpretation to the use of neural networks in texture classification.
Sparse coding has been incorporated in models of the visual cortex for its computational advantages and connection to biology. But how the level of sparsity contributes to performance on visual tasks is not well understood. In this work, sparse coding has been integrated into an existing hierarchical V2 model (Hosoya and Hyv\"arinen, 2015), but replacing its independent component analysis (ICA) with an explicit sparse coding in which the degree of sparsity can be controlled. After training, the sparse coding basis functions with a higher degree of sparsity resembled qualitatively different structures, such as curves and corners. The contributions of the models were assessed with image classification tasks, specifically tasks associated with mid-level vision including figure-ground classification, texture classification, and angle prediction between two line stimuli. In addition, the models were assessed in comparison to a texture sensitivity measure that has been reported in V2 (Freeman et al., 2013), and a deleted-region inference task. The results from the experiments show that while sparse coding performed worse than ICA at classifying images, only sparse coding was able to better match the texture sensitivity level of V2 and infer deleted image regions, both by increasing the degree of sparsity in sparse coding. Higher degrees of sparsity allowed for inference over larger deleted image regions. The mechanism that allows for this inference capability in sparse coding is described here.
To realize accurate texture classification, this article proposes a complex networks (CN)-based multi-feature fusion method to recognize texture images. Specifically, we propose two feature extractors to detect the global and local features of texture images respectively. To capture the global features, we first map a texture image as an undirected graph based on pixel location and intensity, and three feature measurements are designed to further decipher the image features, which retains the image information as much as possible. Then, given the original band images (BI) and the generated feature images, we encode them based on the local binary patterns (LBP). Therefore, the global feature vector is obtained by concatenating four spatial histograms. To decipher the local features, we jointly transfer and fine-tune the pre-trained VGGNet-16 model. Next, we fuse and connect the middle outputs of max-pooling layers (MP), and generate the local feature vector by a global average pooling layer (GAP). Finally, the global and local feature vectors are concatenated to form the final feature representation of texture images. Experiment results show that the proposed method outperforms the state-of-the-art statistical descriptors and the deep convolutional neural networks (CNN) models.
Objective is to assess the ability of texture features for detecting radiographic patellofemoral osteoarthritis (PFOA) from knee lateral view radiographs. We used lateral view knee radiographs from MOST public use datasets (n = 5507 knees). Patellar region-of-interest (ROI) was automatically detected using landmark detection tool (BoneFinder). Hand-crafted features, based on LocalBinary Patterns (LBP), were then extracted to describe the patellar texture. First, a machine learning model (Gradient Boosting Machine) was trained to detect radiographic PFOA from the LBP features. Furthermore, we used end-to-end trained deep convolutional neural networks (CNNs) directly on the texture patches for detecting the PFOA. The proposed classification models were eventually compared with more conventional reference models that use clinical assessments and participant characteristics such as age, sex, body mass index(BMI), the total WOMAC score, and tibiofemoral Kellgren-Lawrence (KL) grade. Atlas-guided visual assessment of PFOA status by expert readers provided in the MOST public use datasets was used as a classification outcome for the models. Performance of prediction models was assessed using the area under the receiver operating characteristic curve (ROC AUC), the area under the precision-recall (PR) curve-average precision (AP)-, and Brier score in the stratified 5-fold cross validation setting.Of the 5507 knees, 953 (17.3%) had PFOA. AUC and AP for the strongest reference model including age, sex, BMI, WOMAC score, and tibiofemoral KL grade to predict PFOA were 0.817 and 0.487, respectively. Textural ROI classification using CNN significantly improved the prediction performance (ROC AUC= 0.889, AP= 0.714). We present the first study that analyses patellar bone texture for diagnosing PFOA. Our results demonstrates the potential of using texture features of patella to predict PFOA.
To overcome the limitations of original local binary patterns (LBP), this article proposes a new texture descriptor aided by complex networks (CN) and LBP, named CN-LBP. Specifically, we first abstract a texture image (TI) as directed graphs over different bands with the help of pixel distance, intensity, and gradient (magnitude and angle). Second, several CN-based feature measurements (including clustering coefficient, in-degree centrality, out-degree centrality, and eigenvector centrality) are selected to further decipher the texture features, which generates four feature images that can retain the image information as much as possible. Third, given the original TIs, gradient images (GI), and generated feature images, we can obtain the discriminative representation of texture images based on uniform LBP (ULBP). Finally, the feature vector is obtained by jointly calculating and concatenating the spatial histograms. In contrast to original LBP, the proposed texture descriptor contains more detailed image information, and shows resistance to imaging and noise. Experiment results on four datasets demonstrate that the proposed texture descriptor can significantly improve the classification accuracies compared with the state-of-the-art LBP-based variants and deep learning-based methods.
Elliptically-contoured distributions (ECD) play a significant role, in computer vision, image processing, radar, and biomedical signal processing. Maximum likelihood. estimation (MLE) of ECD leads to a system of non-linear equations, most-often addressed using fixed-point (FP) methods. Unfortunately, the computation time required for these methods is unacceptably long, for large-scale or high-dimensional datasets. To overcome this difficulty, the present work introduces a Riemannian optimisation method, the information stochastic gradient (ISG). The ISG is an online (recursive) method, which achieves the same performance as MLE, for large-scale datasets, while requiring modest memory and time resources. To develop the ISG method, the Riemannian information gradient is derived taking into account the product manifold associated to the underlying parameter space of the ECD. From this information gradient definition, we define also, the information deterministic gradient (IDG), an offline (batch) method, which is an alternative, for moderate-sized datasets. The present work formulates these two methods, and demonstrates their performance through numerical simulations. Two applications, to image re-colorization, and to texture classification, are also worked out.
The reference frame memory accesses in inter prediction result in high DRAM bandwidth requirement and power consumption. This problem is more intensive by the adoption of intra block copy (IBC), a new coding tool in the screen content coding (SCC) extension to High Efficiency Video Coding (HEVC). In this paper, we propose a lossless recompression scheme that compresses the reference blocks in intra prediction, i.e., intra block copy, before storing them into DRAM to alleviate this problem. The proposal performs pixel-wise texture analysis with an edge-based adaptive prediction method yet no signaling for direction in bitstreams, thus achieves a high gain for compression. Experimental results demonstrate that the proposed scheme shows a 72% data reduction rate on average, which solves the memory bandwidth problem.
The continuous online monitoring of early signs of plant and crop diseases, at their early stages before a potential spread, is of high importance and necessitates multi-disciplinary techniques. Within this study a proposed technique achieves this goal by exploiting laser physics, textural image analysis, and AI for Shot hole disease. In this technique, specific laser light with a wavelength shorter than a sub-cellular component of an inspected plant, produces an interaction within the sub-cellular components and generates laser speckle patterns which can characterize those specific plant cells' features. The generated laser speckle image data then be quantized by texture analysis and classified by Bayesian networks. Such comparative methods manage to detect the differences at sub-cellular scales, such as nuclei modification, cellular shape, or size deformation, etc. for Shot hole disease with high classification accuracy between the healthy and diseased plants. The technique is capable of continuous online observation and monitoring of the plant or crop diseases via a wireless network at low instrumental cost and may replace the costly ground-truth field works
Passive radio frequency (RF) sensing and monitoring of human daily activities in elderly care homes is an emerging topic. Micro-Doppler radars are an appealing solution considering their non-intrusiveness, deep penetration, and high-distance range. Unsupervised activity recognition using Doppler radar data has not received attention, in spite of its importance in case of unlabelled or poorly labelled activities in real scenarios. This study proposes two unsupervised feature extraction methods for the purpose of human activity monitoring using Doppler-streams. These include a local Discrete Cosine Transform (DCT)-based feature extraction method and a local entropy-based feature extraction method. In addition, a novel application of Convolutional Variational Autoencoder (CVAE) feature extraction is employed for the first time for Doppler radar data. The three feature extraction architectures are compared with the previously used Convolutional Autoencoder (CAE) and linear feature extraction based on Principal Component Analysis (PCA) and 2DPCA. Unsupervised clustering is performed using K-Means and K-Medoids. The results show the superiority of DCT-based method, entropy-based method, and CVAE features compared to CAE, PCA, and 2DPCA, with more than 5\%-20\% average accuracy. In regards to computation time, the two proposed methods are noticeably much faster than the existing CVAE. Furthermore, for high-dimensional data visualisation, three manifold learning techniques are considered. The methods are compared for the projection of raw data as well as the encoded CVAE features. All three methods show an improved visualisation ability when applied to the encoded CVAE features.
Temporal interpolation has the potential to be a powerful tool for video compression. Existing methods for frame interpolation do not discriminate between video textures and generally invoke a single general model capable of interpolating a wide range of video content. However, past work on video texture analysis and synthesis has shown that different textures exhibit vastly different motion characteristics and they can be divided into three classes (static, dynamic continuous and dynamic discrete). In this work, we study the impact of video textures on video frame interpolation, and propose a novel framework where, given an interpolation algorithm, separate models are trained on different textures. Our study shows that video texture has significant impact on the performance of frame interpolation models and it is beneficial to have separate models specifically adapted to these texture classes, instead of training a single model that tries to learn generic motion. Our results demonstrate that models fine-tuned using our framework achieve, on average, a 0.3dB gain in PSNR on the test set used.
Identifying the origin of a sample image in biometric systems can be beneficial for data authentication in case of attacks against the system and for initiating sensor-specific processing pipelines in sensor-heterogeneous environments. Motivated by shortcomings of the photo response non-uniformity (PRNU) based method in the biometric context, we use a texture classification approach to detect the origin of finger vein sample images. Based on eight publicly available finger vein datasets and applying eight classical yet simple texture descriptors and SVM classification, we demonstrate excellent sensor model identification results for raw finger vein samples as well as for the more challenging region of interest data. The observed results establish texture descriptors as effective competitors to PRNU in finger vein sensor model identification.
In this work, we present an in-depth and systematic analysis using tools such as local interpretable model-agnostic explanations (LIME) (arXiv:1602.04938) and divergence measures to analyze what changes lead to improvement in performance in fine tuned models for synthetic aperture sonar (SAS) data. We examine the sensitivity to factors in the fine tuning process such as class imbalance. Our findings show not only an improvement in seafloor texture classification, but also provide greater insight into what features play critical roles in improving performance as well as a knowledge of the importance of balanced data for fine tuning deep learning models for seafloor classification in SAS imagery.
In this paper, a smile and laugh facial expression is presented based on dimension reduction and description process of the key points. The paper has two main objectives; the first is to extract the local critical points in terms of their apparent features, and the second is to reduce the system's dependence on training inputs. To achieve these objectives, three different scenarios on extracting the features are proposed. First of all, the discrete parts of a face are detected by local binary pattern method that is used to extract a set of global feature vectors for texture classification considering various regions of an input-image face. Then, in the first scenario and with respect to the correlation changes of adjacent pixels on the texture of a mouth area, a set of local key points are extracted using the Harris corner detector. In the second scenario, the dimension reduction of the extracted points of first scenario provided by principal component analysis algorithm leading to reduction in computational costs and overall complexity without loss of performance and flexibility, etc.
In this paper, we propose a new texture descriptor, completed local derivative pattern (CLDP). In contrast to completed local binary pattern (CLBP), which involves only local differences at each scale, CLDP encodes the directional variation of the local differences of two scales as a complementary component to local patterns in CLBP. The new component in CLDP, with regarded as the directional derivative pattern, reflects the directional smoothness of local textures without increasing computation complexity. Experimental results on the Outex database show that CLDP, as a uni-scale pattern, outperforms uni-scale state-of-the-art texture descriptors on texture classification and has comparable performance with multi-scale texture descriptors.
In this paper, we propose a new texture descriptor, scale selective extended local binary pattern (SSELBP), to characterize texture images with scale variations. We first utilize multi-scale extended local binary patterns (ELBP) with rotation-invariant and uniform mappings to capture robust local micro- and macro-features. Then, we build a scale space using Gaussian filters and calculate the histogram of multi-scale ELBPs for the image at each scale. Finally, we select the maximum values from the corresponding bins of multi-scale ELBP histograms at different scales as scale-invariant features. A comprehensive evaluation on public texture databases (KTH-TIPS and UMD) shows that the proposed SSELBP has high accuracy comparable to state-of-the-art texture descriptors on gray-scale-, rotation-, and scale-invariant texture classification but uses only one-third of the feature dimension.
With the increase in interest in deployment of robots in unstructured environments to work alongside humans, the development of human-like sense of touch for robots becomes important. In this work, we implement a multi-channel neuromorphic tactile system that encodes contact events as discrete spike events that mimic the behavior of slow adapting mechanoreceptors. We study the impact of information pooling across artificial mechanoreceptors on classification performance of spatially non-uniform naturalistic textures. We encoded the spatio-temporal activation patterns of mechanoreceptors through gray-level co-occurrence matrix computed from time-varying mean spiking rate-based tactile response volume. We found that this approach greatly improved texture classification in comparison to use of individual mechanoreceptor response alone. In addition, the performance was also more robust to changes in sliding velocity. The importance of exploiting precise spatial and temporal correlations between sensory channels is evident from the fact that on either removal of precise temporal information or altering of spatial structure of response pattern, a significant performance drop was observed. This study thus demonstrates the superiority of population coding approaches that can exploit the precise spatio-temporal information encoded in activation patterns of mechanoreceptor populations. It, therefore, makes an advance in the direction of development of bio-inspired tactile systems required for realistic touch applications in robotics and prostheses.