We aim at constructing a high performance model for defect detection that detects unknown anomalous patterns of an image without anomalous data. To this end, we propose a two-stage framework for building anomaly detectors using normal training data only. We first learn self-supervised deep representations and then build a generative one-class classifier on learned representations. We learn representations by classifying normal data from the CutPaste, a simple data augmentation strategy that cuts an image patch and pastes at a random location of a large image. Our empirical study on MVTec anomaly detection dataset demonstrates the proposed algorithm is general to be able to detect various types of real-world defects. We bring the improvement upon previous arts by 3.1 AUCs when learning representations from scratch. By transfer learning on pretrained representations on ImageNet, we achieve a new state-of-theart 96.6 AUC. Lastly, we extend the framework to learn and extract representations from patches to allow localizing defective areas without annotations during training.
Deep learning is now the gold standard in computer vision-based quality inspection systems. In order to detect defects, supervised learning is often utilized, but necessitates a large amount of annotated images, which can be costly: collecting, cleaning, and annotating the data is tedious and limits the speed at which a system can be deployed as everything the system must detect needs to be observed first. This can impede the inspection of rare defects, since very few samples can be collected by the manufacturer. In this work, we focus on simulations to solve this issue. We first present a generic simulation pipeline to render images of defective or healthy (non defective) parts. As metallic parts can be highly textured with small defects like holes, we design a texture scanning and generation method. We assess the quality of the generated images by training deep learning networks and by testing them on real data from a manufacturer. We demonstrate that we can achieve encouraging results on real defect detection using purely simulated data. Additionally, we are able to improve global performances by concatenating simulated and real data, showing that simulations can complement real images to boost performances. Lastly, using domain adaptation techniques helps improving slightly our final results.
This paper introduces an improved image processing method usable in capacitive imaging applications. Standard capacitive imaging tends to prefer amplitude-based images over the use of phase due to better signal-to-noise ratios. The new approach exploits the best features of both types of information by combining them to form clearer images, hence improving both defect detection and characterization in non-destructive evaluation. The methodology is demonstrated and optimized using a benchmark sample. Additional experiments on glass fibre composite sample illustrate the advantages of the technique.
Quality inspection applications in industry are required to move towards a zero-defect manufacturing scenario, withnon-destructive inspection and traceability of 100 % of produced parts. Developing robust fault detection and classification modelsfrom the start-up of the lines is challenging due to the difficulty in getting enough representative samples of the faulty patternsand the need to manually label them. This work presents a methodology to develop a robust inspection system, targeting thesepeculiarities, in the context of solar cell manufacturing. The methodology is divided into two phases: In the first phase, an anomalydetection model based on a Generative Adversarial Network (GAN) is employed. This model enables the detection and localizationof anomalous patterns within the solar cells from the beginning, using only non-defective samples for training and without anymanual labeling involved. In a second stage, as defective samples arise, the detected anomalies will be used as automaticallygenerated annotations for the supervised training of a Fully Convolutional Network that is capable of detecting multiple types offaults. The experimental results using 1873 EL images of monocrystalline cells show that (a) the anomaly detection scheme can beused to start detecting features with very little available data, (b) the anomaly detection may serve as automatic labeling in order totrain a supervised model, and (c) segmentation and classification results of supervised models trained with automatic labels arecomparable to the ones obtained from the models trained with manual labels.
Although training data is essential for machine learning, railway companies are facing difficulties in gathering adequate images of defective equipment due to their proactive replacement of would be defective equipment. Nevertheless, proactive replacement is indispensable for safe and undisturbed operation of public transport. In this research, we have developed a model using CycleGAN to generate artificial images of defective equipment instead of real images. By adopting these generated images as training data, we verified that these images are indistinguishable from real images and they play a vital role in enhancing the accuracy of the defect detection models.
Benchmark datasets have a significant impact on accelerating research in programming language tasks. In this paper, we introduce CodeXGLUE, a benchmark dataset to foster machine learning research for program understanding and generation. CodeXGLUE includes a collection of 10 tasks across 14 datasets and a platform for model evaluation and comparison. CodeXGLUE also features three baseline systems, including the BERT-style, GPT-style, and Encoder-Decoder models, to make it easy for researchers to use the platform. The availability of such data and baselines can help the development and validation of new methods that can be applied to various program understanding and generation problems.
Defect detection in the manufacturing industry is of utmost importance for product quality inspection. Recently, optical defect detection has been investigated as an anomaly detection using different deep learning methods. However, the recent works do not explore the use of point pattern features, such as SIFT for anomaly detection using the recently developed set-based methods. In this paper, we present an evaluation of different point pattern feature detectors and descriptors for defect detection application. The evaluation is performed within the random finite set framework. Handcrafted point pattern features, such as SIFT as well as deep features are used in this evaluation. Random finite set-based defect detection is compared with state-of-the-arts anomaly detection methods. The results show that using point pattern features, such as SIFT as data points for random finite set-based anomaly detection achieves the most consistent defect detection accuracy on the MVTec-AD dataset.
Vision-based detection on surface defects has long postulated in the magnetic tile automation process. In this work, we introduce a real-time and multi-module neural network model called MCuePush U-Net, specifically designed for the image saliency detection of magnetic tile. We show that the model exceeds the state-of-the-art, in which it both effectively and explicitly maps multiple surface defects from low-contrast images. Our model significantly reduces time cost of machinery from 0.5s per image to 0.07s, and enhances saliency accuracy on surface defect detection.
Segmentation of additive manufacturing (AM) defects in X-ray Computed Tomography (XCT) images is challenging, due to the poor contrast, small sizes and variation in appearance of defects. Automatic segmentation can, however, provide quality control for additive manufacturing. Over recent years, three-dimensional convolutional neural networks (3D CNNs) have performed well in the volumetric segmentation of medical images. In this work, we leverage techniques from the medical imaging domain and propose training a 3D U-Net model to automatically segment defects in XCT images of AM samples. This work not only contributes to the use of machine learning for AM defect detection but also demonstrates for the first time 3D volumetric segmentation in AM. We train and test with three variants of the 3D U-Net on an AM dataset, achieving a mean intersection of union (IOU) value of 88.4%.
Visual steel surface defect detection is an essential step in steel sheet manufacturing. Several machine learning-based automated visual inspection (AVI) methods have been studied in recent years. However, most steel manufacturing industries still use manual visual inspection due to training time and inaccuracies involved with AVI methods. Automatic steel defect detection methods could be useful in less expensive and faster quality control and feedback. But preparing the annotated training data for segmentation and classification could be a costly process. In this work, we propose to use the Transfer Learning-based U-Net (TLU-Net) framework for steel surface defect detection. We use a U-Net architecture as the base and explore two kinds of encoders: ResNet and DenseNet. We compare these nets' performance using random initialization and the pre-trained networks trained using the ImageNet data set. The experiments are performed using Severstal data. The results demonstrate that the transfer learning performs 5% (absolute) better than that of the random initialization in defect classification. We found that the transfer learning performs 26% (relative) better than that of the random initialization in defect segmentation. We also found the gain of transfer learning increases as the training data decreases, and the convergence rate with transfer learning is better than that of the random initialization.
With the rapid growth of display devices, quality inspection via machine vision technology has become increasingly important for flat-panel displays (FPD) industry. This paper discloses a novel visual inspection system for liquid crystal display (LCD), which is currently a dominant type in the FPD industry. The system is based on two cornerstones: robust/high-performance defect recognition model and cognitive visual inspection service architecture. A hybrid application of conventional computer vision technique and the latest deep convolutional neural network (DCNN) leads to an integrated defect detection, classfication and impact evaluation model that can be economically trained with only image-level class annotations to achieve a high inspection accuracy. In addition, the properly trained model is robust to the variation of the image qulity, significantly alleviating the dependency between the model prediction performance and the image aquisition environment. This in turn justifies the decoupling of the defect recognition functions from the front-end device to the back-end serivce, motivating the design and realization of the cognitive visual inspection service architecture. Empirical case study is performed on a large-scale real-world LCD dataset from a manufacturing line with different layers and products, which shows the promising utility of our system, which has been deployed in a real-world LCD manufacturing line from a major player in the world.
Automatic defect detection is a challenging task because of the variability in texture and type of fabric defects. An effective defect detection system enables manufacturers to improve the quality of processes and products. Automation across the textile manufacturing systems would reduce fabric wastage and increase profitability by saving cost and resources. There are different contemporary research on automatic defect detection systems using image processing and machine learning techniques. These techniques differ from each other based on the manufacturing processes and defect types. Researchers have also been able to establish real-time defect detection system during weaving. Although, there has been research on patterned fabric defect detection, these defects are related to weaving faults such as holes, and warp and weft defects. But, there has not been any research that is designed to detect defects that arise during such as spot and print mismatch. This research has fulfilled this gap by developing a print fabric database and implementing deep convolutional neural network (CNN).
We present three parallel-beam tomographic datasets of 94 apples with internal defects along with defect label files. The datasets are prepared for development and testing of data-driven, learning-based image reconstruction, segmentation and post-processing methods. The three versions are a noiseless simulation; simulation with added Gaussian noise, and with scattering noise. The datasets are based on real 3D X-ray CT data and their subsequent volume reconstructions. The ground truth images, based on the volume reconstructions, are also available through this project. Apples contain various defects, which naturally introduce a label bias. We tackle this by formulating the bias as an optimization problem. In addition, we demonstrate solving this problem with two methods: a simple heuristic algorithm and through mixed integer quadratic programming. This ensures the datasets can be split into test, training or validation subsets with the label bias eliminated. Therefore the datasets can be used for image reconstruction, segmentation, automatic defect detection, and testing the effects of (as well as applying new methodologies for removing) label bias in machine learning.
The multi-scale defect detection for photovoltaic (PV) cell electroluminescence (EL) images is a challenging task, due to the feature vanishing as network deepens. To address this problem, an attention-based top-down and bottom-up architecture is developed to accomplish multi-scale feature fusion. This architecture, called Bidirectional Attention Feature Pyramid Network (BAFPN), can make all layers of the pyramid share similar semantic features. In BAFPN, cosine similarity is employed to measure the importance of each pixel in the fused features. Furthermore, a novel object detector is proposed, called BAF-Detector, which embeds BAFPN into Region Proposal Network (RPN) in Faster RCNN+FPN. BAFPN improves the robustness of the network to scales, thus the proposed detector achieves a good performance in multi-scale defects detection task. Finally, the experimental results on a large-scale EL dataset including 3629 images, 2129 of which are defective, show that the proposed method achieves 98.70% (F-measure), 88.07% (mAP), and 73.29% (IoU) in terms of multi-scale defects classification and detection results in raw PV cell EL images.
Visual defect detection is critical to ensure the quality of most products. However, the majority of small and medium-sized manufacturing enterprises still rely on tedious and error-prone human manual inspection. The main reasons include: 1) the existing automated visual defect detection systems require altering production assembly lines, which is time consuming and expensive 2) the existing systems require manually collecting defective samples and labeling them for a comparison-based algorithm or training a machine learning model. This introduces a heavy burden for small and medium-sized manufacturing enterprises as defects do not happen often and are difficult and time-consuming to collect. Furthermore, we cannot exhaustively collect or define all defect types as any new deviation from acceptable products are defects. In this paper, we overcome these challenges and design a three-stage plug-and-play fully automated unsupervised 360-degree defect detection system. In our system, products are freely placed on an unaltered assembly line and receive 360 degree visual inspection with multiple cameras from different angles. As such, the images collected from real-world product assembly lines contain lots of background noise. The products face different angles. The product sizes vary due to the distance to cameras. All these make defect detection much more difficult. Our system use object detection, background subtraction and unsupervised normalizing flow-based defect detection techniques to tackle these difficulties. Experiments show our system can achieve 0.90 AUROC in a real-world non-altered drinkware production assembly line.
Fabric defect detection is a crucial quality control step in the textile manufacturing industry. In this article, machine vision system based on the Sylvester Matrix Based Similarity Method (SMBSM) is proposed to automate the defect detection process. The algorithm involves six phases, namely resolution matching, image enhancement using Histogram Specification and Median-Mean Based Sub-Image-Clipped Histogram Equalization, image registration through alignment and hysteresis process, image subtraction, edge detection, and fault detection by means of the rank of the Sylvester matrix. The experimental results demonstrate that the proposed method is robust and yields an accuracy of 93.4%, precision of 95.8%, with 2275 ms computational speed.
We present a compressive sensing based defect detection by multiple input multiple output (MIMO) wireless radar. Here, defects are inside a layered material structure, therefore, due to reflections from the surface of the layered material structure the defect detection is challenging. By utilizing a low-rank nature of the reflections of the layered material structure and sparse nature of the defects, we propose a method based on rank minimization and sparse recovery. To improve the accuracy in the recovery of low-rank and sparse components, we propose a non-convex approach based on the iteratively reweighted nuclear norm and iteratively reweighted $\ell_1-$norm algorithm. Our numerical results show that the proposed method is able to demix and recover the signalling responses of the defects and layered structure successfully from substantially reduced number of observations. Further, the proposed approach outperforms the state-of-the-art clutter reduction approaches
The Direct Structured-Light Inspection Technique (DSIT) proposed in this paper is a novel method that can be implemented under two types of binary structured light illumination to detect contaminants and defects on specular surfaces and transparent objects, in which light reflection system is used to detect specular surfaces, while light transmission system is applied for transparent object inspection. Based on this technique, contaminant and defect distribution can be directly obtained without any calculation process. Relevant simulations and experiments are performed to prove the effectiveness of DSIT.
Modern industry requires modern solutions for monitoring the automatic production of goods. Smart monitoring of the functionality of the mechanical parts of technology systems or machines is mandatory for a fully automatic production process. Although Deep Learning has been advancing, allowing for real-time object detection and other tasks, little has been investigated about the effectiveness of specially designed Convolutional Neural Networks for defect detection and industrial object recognition. In the particular study, we employed six publically available industrial-related datasets containing defect materials and industrial tools or engine parts, aiming to develop a specialized model for pattern recognition. Motivated by the recent success of the Virtual Geometry Group (VGG) network, we propose a modified version of it, called Multipath VGG19, which allows for more local and global feature extraction, while the extra features are fused via concatenation. The experiments verified the effectiveness of MVGG19 over the traditional VGG19. Specifically, top classification performance was achieved in five of the six image datasets, while the average classification improvement was 6.95%.
Active and passive thermography are two efficient techniques extensively used to measure heterogeneous thermal patterns leading to subsurface defects for diagnostic evaluations. This study conducts a comparative analysis on low-rank matrix approximation methods in thermography with applications of semi-, convex-, and sparse- non-negative matrix factorization (NMF) methods for detecting subsurface thermal patterns. These methods inherit the advantages of principal component thermography (PCT) and sparse PCT, whereas tackle negative bases in sparse PCT with non-negative constraints, and exhibit clustering property in processing data. The practicality and efficiency of these methods are demonstrated by the experimental results for subsurface defect detection in three specimens (for different depth and size defects) and preserving thermal heterogeneity for distinguishing breast abnormality in breast cancer screening dataset (accuracy of 74.1%, 75.8%, and 77.8%).