Prediction of genetic biomarkers, e.g., microsatellite instability and BRAF in colorectal cancer is crucial for clinical decision making. In this paper, we propose a whole slide image (WSI) based genetic biomarker prediction method via prompting techniques. Our work aims at addressing the following challenges: (1) extracting foreground instances related to genetic biomarkers from gigapixel WSIs, and (2) the interaction among the fine-grained pathological components in WSIs.Specifically, we leverage large language models to generate medical prompts that serve as prior knowledge in extracting instances associated with genetic biomarkers. We adopt a coarse-to-fine approach to mine biomarker information within the tumor microenvironment. This involves extracting instances related to genetic biomarkers using coarse medical prior knowledge, grouping pathology instances into fine-grained pathological components and mining their interactions. Experimental results on two colorectal cancer datasets show the superiority of our method, achieving 91.49% in AUC for MSI classification. The analysis further shows the clinical interpretability of our method. Code is publicly available at https://github.com/DeepMed-Lab-ECNU/PromptBio.
Recent advancements in medical vision-language pre-training (MedVLP) have significantly enhanced zero-shot medical vision tasks such as image classification by leveraging large-scale medical image-text pair pre-training. However, the performance of these tasks can be heavily influenced by the variability in textual prompts describing the categories, necessitating robustness in MedVLP models to diverse prompt styles. Yet, this sensitivity remains underexplored. In this work, we are the first to systematically assess the sensitivity of three widely-used MedVLP methods to a variety of prompts across 15 different diseases. To achieve this, we designed six unique prompt styles to mirror real clinical scenarios, which were subsequently ranked by interpretability. Our findings indicate that all MedVLP models evaluated show unstable performance across different prompt styles, suggesting a lack of robustness. Additionally, the models' performance varied with increasing prompt interpretability, revealing difficulties in comprehending complex medical concepts. This study underscores the need for further development in MedVLP methodologies to enhance their robustness to diverse zero-shot prompts.
Vision-language model (VLM) embeddings have been shown to encode biases present in their training data, such as societal biases that prescribe negative characteristics to members of various racial and gender identities. VLMs are being quickly adopted for a variety of tasks ranging from few-shot classification to text-guided image generation, making debiasing VLM embeddings crucial. Debiasing approaches that fine-tune the VLM often suffer from catastrophic forgetting. On the other hand, fine-tuning-free methods typically utilize a "one-size-fits-all" approach that assumes that correlation with the spurious attribute can be explained using a single linear direction across all possible inputs. In this work, we propose Bend-VLM, a nonlinear, fine-tuning-free approach for VLM embedding debiasing that tailors the debiasing operation to each unique input. This allows for a more flexible debiasing approach. Additionally, we do not require knowledge of the set of inputs a priori to inference time, making our method more appropriate for online, open-set tasks such as retrieval and text guided image generation.
자동화된 상품 인식은 현대 리테일 인텔리전스의 핵심이다. 그러나 방대한 기업 카탈로그에 대해 실세계 매장 이미지를 정확하게 매칭하는 것은 대규모 응용에서 여전히 주요한 확장성 병목으로 남아 있다. 본 연구에서는 이 과제를 표준적인 폐쇄형 집합 분류(classification) 작업이 아닌 임베딩(embedding) 기반 교차 도메인 검색 문제로 재정의함으로써 이 문제에 접근한다. 구체적으로, 광범위한 인벤토리로부터 주어진 실세계 상품 쿼리 크롭에 대해 가장 대응되는 카탈로그 참조 이미지를 검색하는 것을 목표로 정의한다. 깔끔한 스튜디오 팩샷과 노이즈가 있는 매장 내 쿼리 사이의 심각한 도메인 격차를 해소하기 위해, 본 논문은 새로운 카탈로그-실세계 다단계 대조 학습 패러다임(Cat2Real)을 제안한다. 이 프레임워크는 아이템 수준 및 이미지 수준 유사도를 체계적으로 활용하여 타깃화된 하드 네거티브 마이닝을 유도함으로써 비전 백본을 파인튜닝한다. 광범위한 실증적 평가를 통해, 제안된 패러다임이 미학습 상품 및 카테고리로 원활하게 확장됨을 입증하며, 새로운 인벤토리에 대한 실세계 학습(training) 이미지가 전혀 없는 상황에서도 뛰어난 제로샷(zero-shot) 일반화 성능을 달성함을 보인다.
Neural net classifiers trained on data with annotated class labels can also capture apparent visual similarity among categories without being directed to do so. We study whether this observation can be extended beyond the conventional domain of supervised learning: Can we learn a good feature representation that captures apparent similarity among instances, instead of classes, by merely asking the feature to be discriminative of individual instances? We formulate this intuition as a non-parametric classification problem at the instance-level, and use noise-contrastive estimation to tackle the computational challenges imposed by the large number of instance classes. Our experimental results demonstrate that, under unsupervised learning settings, our method surpasses the state-of-the-art on ImageNet classification by a large margin. Our method is also remarkable for consistently improving test performance with more training data and better network architectures. By fine-tuning the learned feature, we further obtain competitive results for semi-supervised learning and object detection tasks. Our non-parametric model is highly compact: With 128 features per image, our method requires only 600MB storage for a million images, enabling fast nearest neighbour retrieval at the run time.
Transfer learning for feature extraction can be used to exploit deep representations in contexts where there is very few training data, where there are limited computational resources, or when tuning the hyper-parameters needed for training is not an option. While previous contributions to feature extraction propose embeddings based on a single layer of the network, in this paper we propose a full-network embedding which successfully integrates convolutional and fully connected features, coming from all layers of a deep convolutional neural network. To do so, the embedding normalizes features in the context of the problem, and discretizes their values to reduce noise and regularize the embedding space. Significantly, this also reduces the computational cost of processing the resultant representations. The proposed method is shown to outperform single layer embeddings on several image classification tasks, while also being more robust to the choice of the pre-trained model used for obtaining the initial features. The performance gap in classification accuracy between thoroughly tuned solutions and the full-network embedding is also reduced, which makes of the proposed approach a competitive solution for a large set of applications.
The early detection of skin cancer substantially improves the five-year survival rate of patients. It is often difficult to distinguish early malignant tumors from skin images, even by expert dermatologists. Therefore, several classification methods of dermatoscopic images have been proposed, but they have been found to be inadequate or defective for skin cancer detection, and often require a large amount of calculations. This study proposes an improved capsule network called FixCaps for dermoscopic image classification. FixCaps has a larger receptive field than CapsNets by applying a high-performance large-kernel at the bottom convolution layer whose kernel size is as large as 31×31 , in contrast to commonly used 9×9 . The convolutional block attention module was used to reduce the losses of spatial information caused by convolution and pooling. The group convolution was used to avoid model underfitting in the capsule layer. The network can improve the detection accuracy and reduce a great amount of calculations, compared with several existing methods. The experimental results showed that FixCaps is better than IRv2-SA for skin cancer diagnosis, which achieved an accuracy of 96.49% on the HAM10000 dataset.
Liver tumor segmentation and classification are important tasks in computer aided diagnosis. We aim to address three problems: liver tumor screening and preliminary diagnosis in non-contrast computed tomography (CT), and differential diagnosis in dynamic contrast-enhanced CT. A novel framework named Pixel-Lesion-pAtient Network (PLAN) is proposed. It uses a mask transformer to jointly segment and classify each lesion with improved anchor queries and a foreground-enhanced sampling loss. It also has an image-wise classifier to effectively aggregate global information and predict patient-level diagnosis. A large-scale multi-phase dataset is collected containing 939 tumor patients and 810 normal subjects. 4010 tumor instances of eight types are extensively annotated. On the non-contrast tumor screening task, PLAN achieves 95% and 96% in patient-level sensitivity and specificity. On contrast-enhanced CT, our lesion-level detection precision, recall, and classification accuracy are 92%, 89%, and 86%, outperforming widely used CNN and transformers for lesion segmentation. We also conduct a reader study on a holdout set of 250 cases. PLAN is on par with a senior human radiologist, showing the clinical significance of our results.
We present a tree-structured network architecture for large scale image classification. The trunk of the network contains convolutional layers optimized over all classes. At a given depth, the trunk splits into separate branches, each dedicated to discriminate a different subset of classes. Each branch acts as an expert classifying a set of categories that are difficult to tell apart, while the trunk provides common knowledge to all experts in the form of shared features. The training of our "network of experts" is completely end-to-end: the partition of categories into disjoint subsets is learned simultaneously with the parameters of the network trunk and the experts are trained jointly by minimizing a single learning objective over all classes. The proposed structure can be built from any existing convolutional neural network (CNN). We demonstrate its generality by adapting 4 popular CNNs for image categorization into the form of networks of experts. Our experiments on CIFAR100 and ImageNet show that in every case our method yields a substantial improvement in accuracy over the base CNN, and gives the best result achieved so far on CIFAR100. Finally, the improvement in accuracy comes at little additional cost: compared to the base network, the training time is only moderately increased and the number of parameters is comparable or in some cases even lower.
Zero-shot learning (ZSL) aims to train a model on seen classes and recognize unseen classes by knowledge transfer through shared auxiliary information. Recent studies reveal that documents from encyclopedias provide helpful auxiliary information. However, existing methods align noisy documents, entangled in visual and non-visual descriptions, with image regions, yet solely depend on implicit learning. These models fail to filter non-visual noise reliably and incorrectly align non-visual words to image regions, which is harmful to knowledge transfer. In this work, we propose a novel multi-attribute document supervision framework to remove noises at both document collection and model learning stages. With the help of large language models, we introduce a novel prompt algorithm that automatically removes non-visual descriptions and enriches less-described documents in multiple attribute views. Our proposed model, MADS, extracts multi-view transferable knowledge with information decoupling and semantic interactions for semantic alignment at local and global levels. Besides, we introduce a model-agnostic focus loss to explicitly enhance attention to visually discriminative information during training, also improving existing methods without additional parameters. With comparable computation costs, MADS consistently outperforms the SOTA by 7.2% and 8.2% on average in three benchmarks for document-based ZSL and GZSL settings, respectively. Moreover, we qualitatively offer interpretable predictions from multiple attribute views.
Learning medical visual representations from image-report pairs through joint learning has garnered increasing research attention due to its potential to alleviate the data scarcity problem in the medical domain. The primary challenges stem from the lengthy reports that feature complex discourse relations and semantic pathologies. Previous works have predominantly focused on instance-wise or token-wise cross-modal alignment, often neglecting the importance of pathological-level consistency. This paper presents a novel framework PLACE that promotes the Pathological-Level Alignment and enriches the fine-grained details via Correlation Exploration without additional human annotations. Specifically, we propose a novel pathological-level cross-modal alignment (PCMA) approach to maximize the consistency of pathology observations from both images and reports. To facilitate this, a Visual Pathology Observation Extractor is introduced to extract visual pathological observation representations from localized tokens. The PCMA module operates independently of any external disease annotations, enhancing the generalizability and robustness of our methods. Furthermore, we design a proxy task that enforces the model to identify correlations among image patches, thereby enriching the fine-grained details crucial for various downstream tasks. Experimental results demonstrate that our proposed framework achieves new state-of-the-art performance on multiple downstream tasks, including classification, image-to-text retrieval, semantic segmentation, object detection and report generation.
사후(post-hoc) 멀티모달 정렬은 독립적으로 사전 학습된 비전 및 언어 인코더를 고정한 채 서로 연결하며, 쌍 데이터가 부족할 때 공동 사전 학습의 대안이 된다. 핵심적인 설계 과제는 이질적인 특징 공간 전반에서 비교 가능한 표현을 구성하면서, 각 입력 내에서 서로 다른 의미 개념과 관련된 국소 정보를 유지하는 것이다. 우리는 학습 가능한 앵커와 토큰 증거를 이용한 상대 정렬(Relative Alignment with Learnable Anchors and Token Evidence, ReLATE)을 제안하며, 이는 상대 표현을 공동으로 정의하고 각 상대 좌표에 대한 토큰의 기여도를 결정하는 모달리티별 앵커를 학습한다. 이미지 패치와 텍스트 토큰은 각자의 고정된 특징 공간 내에서 앵커와의 유사도로 표현된다. 그런 다음 앵커별 유사도 풀링(ASP)이 앵커마다 별도로 이러한 유사도에 가중치를 부여하고 집계하여 해당 상대 좌표를 구성한다. 샘플 수준의 대칭 대조 목적함수는 인코더를 업데이트하지 않고 앵커만 최적화하여 결과로 얻은 이미지와 텍스트 표현을 정렬한다. 네 가지 비전-언어 과제에 걸친 13개의 일반 도메인 벤치마크 실험에서, 동일한 쌍 데이터 예산 조건에서 비교 대상 사후 정렬 방법들보다 향상된 성능을 보이며, 데이터 규모와 인코더 조합에 따른 추가 평가도 수행한다. 일반 도메인을 넘어, 우리는 특수 도메인 응용으로서 병리학 분야에서 ReLATE를 평가한다. 고정된 병리 이미지 인코더와 생의학 언어 인코더를 정렬함으로써, ReLATE는 세 가지 제로샷 분류 벤치마크에서 균형 정확도 측면에서 평가된 병리 특화 비전-언어 모델과 대등하거나 이를 능가한다.
드롭아웃 정규화는 학습 중 신경망의 일부를 제거하여 과적합을 줄이는 데 널리 사용된다. 합성곱 신경망(CNN)에서는 컷아웃(cutout)이 다소 유사한 목적을 수행한다. 컷아웃은 데이터 증강으로 구현할 수 있다. 원본 학습 이미지는 유지하고, 일부 영역이 제거된 추가 복사본을 만든다. 본 장에서는 고해상도 클래스 활성화 매핑(High-Resolution Class Activation Mapping, HiResCAM)을 이용해 컷아웃 배치를 개선할 수 있는지 검증한다. 네 가지 통제된 학습 조건을 비교한다: 컷아웃 없음, 표준 무작위 컷아웃, 저살리언시 컷아웃, 고살리언시 컷아웃. RawMal-TF 데이터셋의 그레이스케일 멀웨어 이미지(17개 패밀리, 패밀리당 약 1,000개 샘플)로 실험하고, 자연 이미지와의 비교를 위해 잘 알려진 CIFAR-100 데이터셋으로도 실험한다. 모든 실험은 ResNet18 기반이며 학습 에폭은 약 100이다. 컷아웃 실험에서는 컷아웃 영역을 약 5%, 10%, 20%, 30%로 테스트하고, 원본 학습 이미지당 증강 복사본 수 M∈{4,8}을 고려한다. RawMal-TF 결과는 컷아웃 없음과 비교할 때 무작위, 고살리언시, 저살리언시 세 가지 컷아웃 경우 모두에서 약간 더 나쁘다. 반면 CIFAR-100 실험 결과는 저살리언시 컷아웃에서 약간 개선된다. 이러한 결과는 살리언시 유도 컷아웃의 효과가 도메인에 따라 달라지며, 멀웨어 이미지를 자연 이미지와 동등하게 취급해서는 안 됨을 시사한다.
초분광 원격 탐사는 분광-공간 CNN, Vision Transformer, Mamba, 그래프 신경망(GNN), Kolmogorov-Arnold 네트워크, 자기지도 마스크 오토인코딩을 포함한 다양한 심층 학습(deep learning) 패러다임 전반에서 발전해 왔습니다. 그러나 분산된 저장소, 호환되지 않는 텐서(tensor) 규칙, 비표준화된 평가 방식으로 인해 여전히 진전이 저해되고 있습니다. Hyperspectral Image Models는 공통 레지스트리, 자동 4D/5D 텐서 적응, 표준화된 생성자를 통해 6개 패러다임에 걸친 55개 대표 모델을 통합하는 모듈형 프레임워크로 이러한 과제를 해결합니다. 이 라이브러리는 항공(Airborne), 위성(Spaceborne), UAV, 화성 CRISM 센서의 24개 벤치마크 씬을 통합하며, 캐싱, 레이블 재매핑, PCA, 명시적 밴드 선택 또는 원시 스펙트럼, 선택적 공간 맥스 풀링(pooling), 임의의 PxP 패치 추출을 지원합니다. 겹치는 윈도우로 인한 정확도 과대를 방지하기 위해, 클래스 균형 무작위 분할과 학습-테스트 픽셀 중첩을 제거하는 체비쇼프 가드 밴드(Chebyshev guard band) 기반의 공간적으로 분리된 영역 블로킹을 지원합니다. 실험은 결정론적 시드와 완전한 출처 추적 기능을 갖춘 단일 설정을 사용하여 LaTeX 벤치마크 표와 분류(classification) 맵을 생성합니다. 1,320회의 모델-씬 평가와 6,600회의 시드 기반 실행 결과, 패러다임 평균 간의 차이가 15%p인 것에 비해 평균 정확도가 Botswana의 96.40%에서 Houston 2018의 56.70%에 이르러 아키텍처보다 씬 난이도가 지배적인 영향을 미치는 것으로 나타났습니다. 어떤 패러다임도 보편적으로 우세하지 않았으며, 1M 미만 매개변수(parameter) 모델이 두 자릿수(orders of magnitude) 더 큰 아키텍처와 대등한 성능을 보일 수 있었습니다. 코드는 https://github.com/Tanishq251/Hyperspectral-Image-Models 에서 공개되어 있습니다.
Obtaining a high-quality frontal face image from a low-resolution (LR) non-frontal face image is primarily important for many facial analysis applications. However, mainstreams either focus on super-resolving near-frontal LR faces or frontalizing non-frontal high-resolution (HR) faces. It is desirable to perform both tasks seamlessly for daily-life unconstrained face images. In this paper, we present a novel Vivid Face Hallucination Generative Adversarial Network (VividGAN) for simultaneously super-resolving and frontalizing tiny non-frontal face images. VividGAN consists of coarse-level and fine-level Face Hallucination Networks (FHnet) and two discriminators, i.e., Coarse-D and Fine-D. The coarse-level FHnet generates a frontal coarse HR face and then the fine-level FHnet makes use of the facial component appearance prior, i.e., fine-grained facial components, to attain a frontal HR face image with authentic details. In the fine-level FHnet, we also design a facial component-aware module that adopts the facial geometry guidance as clues to accurately align and merge the frontal coarse HR face and prior information. Meanwhile, two-level discriminators are designed to capture both the global outline of a face image as well as detailed facial characteristics. The Coarse-D enforces the coarsely hallucinated faces to be upright and complete while the Fine-D focuses on the fine hallucinated ones for sharper details. Extensive experiments demonstrate that our VividGAN achieves photo-realistic frontal HR faces, reaching superior performance in downstream tasks, i.e., face recognition and expression classification, compared with other state-of-the-art methods.
Traditional computer vision approaches, based on neural networks (NN), are typically trained on a large amount of image data. By minimizing the cross-entropy loss between a prediction and a given class label, the NN and its visual embedding space are learned to fulfill a given task. However, due to the sole dependence on the image data distribution of the training domain, these models tend to fail when applied to a target domain that differs from their source domain. To learn a more robust NN to domain shifts, we propose the knowledge graph neural network (KG-NN), a neuro-symbolic approach that supervises the training using image-data-invariant auxiliary knowledge. The auxiliary knowledge is first encoded in a knowledge graph with respective concepts and their relationships, which is then transformed into a dense vector representation via an embedding method. Using a contrastive loss function, KG-NN learns to adapt its visual embedding space and thus its weights according to the image-data invariant knowledge graph embedding space. We evaluate KG-NN on visual transfer learning tasks for classification using the mini-ImageNet dataset and its derivatives, as well as road sign recognition datasets from Germany and China. The results show that a visual model trained with a knowledge graph as a trainer outperforms a model trained with cross-entropy in all experiments, in particular when the domain gap increases. Besides better performance and stronger robustness to domain shifts, these KG-NN adapts to multiple datasets and classes without suffering heavily from catastrophic forgetting.
Transfer learning for deep neural networks is the process of first training a base network on a source dataset, and then transferring the learned features (the network's weights) to a second network to be trained on a target dataset. This idea has been shown to improve deep neural network's generalization capabilities in many computer vision tasks such as image recognition and object localization. Apart from these applications, deep Convolutional Neural Networks (CNNs) have also recently gained popularity in the Time Series Classification (TSC) community. However, unlike for image recognition problems, transfer learning techniques have not yet been investigated thoroughly for the TSC task. This is surprising as the accuracy of deep learning models for TSC could potentially be improved if the model is fine-tuned from a pre-trained neural network instead of training it from scratch. In this paper, we fill this gap by investigating how to transfer deep CNNs for the TSC task. To evaluate the potential of transfer learning, we performed extensive experiments using the UCR archive which is the largest publicly available TSC benchmark containing 85 datasets. For each dataset in the archive, we pre-trained a model and then fine-tuned it on the other datasets resulting in 7140 different deep neural networks. These experiments revealed that transfer learning can improve or degrade the model's predictions depending on the dataset used for transfer. Therefore, in an effort to predict the best source dataset for a given target dataset, we propose a new method relying on Dynamic Time Warping to measure inter-datasets similarities. We describe how our method can guide the transfer to choose the best source dataset leading to an improvement in accuracy on 71 out of 85 datasets.
Unsupervised text encoding models have recently fueled substantial progress in NLP. The key idea is to use neural networks to convert words in texts to vector space representations based on word positions in a sentence and their contexts, which are suitable for end-to-end training of downstream tasks. We see a strikingly similar situation in spatial analysis, which focuses on incorporating both absolute positions and spatial contexts of geographic objects such as POIs into models. A general-purpose representation model for space is valuable for a multitude of tasks. However, no such general model exists to date beyond simply applying discretization or feed-forward nets to coordinates, and little effort has been put into jointly modeling distributions with vastly different characteristics, which commonly emerges from GIS data. Meanwhile, Nobel Prize-winning Neuroscience research shows that grid cells in mammals provide a multi-scale periodic representation that functions as a metric for location encoding and is critical for recognizing places and for path-integration. Therefore, we propose a representation learning model called Space2Vec to encode the absolute positions and spatial relationships of places. We conduct experiments on two real-world geographic data for two different tasks: 1) predicting types of POIs given their positions and context, 2) image classification leveraging their geo-locations. Results show that because of its multi-scale representations, Space2Vec outperforms well-established ML approaches such as RBF kernels, multi-layer feed-forward nets, and tile embedding approaches for location modeling and image classification tasks. Detailed analysis shows that all baselines can at most well handle distribution at one scale but show poor performances in other scales. In contrast, Space2Vec's multi-scale representation can handle distributions at different scales.
In many research fields, the sizes of the existing datasets vary widely. Hence, there is a need for machine learning techniques which are well-suited for these different datasets. One possible technique is the self-organizing map (SOM), a type of artificial neural network which is, so far, weakly represented in the field of machine learning. The SOM's unique characteristic is the neighborhood relationship of the output neurons. This relationship improves the ability of generalization on small datasets. SOMs are mostly applied in unsupervised learning and few studies focus on using SOMs as supervised learning approach. Furthermore, no appropriate SOM package is available with respect to machine learning standards and in the widely used programming language Python. In this paper, we introduce the freely available Supervised Self-organizing maps (SuSi) Python package which performs supervised regression and classification. The implementation of SuSi is described with respect to the underlying mathematics. Then, we present first evaluations of the SOM for regression and classification datasets from two different domains of geospatial image analysis. Despite the early stage of its development, the SuSi framework performs well and is characterized by only small performance differences between the training and the test datasets. A comparison of the SuSi framework with existing Python and R packages demonstrates the importance of the SuSi framework. In future work, the SuSi framework will be extended, optimized and upgraded e.g. with tools to better understand and visualize the input data as well as the handling of missing and incomplete data.
Domain adaptation (DA) attempts to transfer the knowledge from a labeled source domain to an unlabeled target domain that follows different distribution from the source. To achieve this, DA methods include a source classification objective to extract the source knowledge and a domain alignment objective to diminish the domain shift, ensuring knowledge transfer. Typically, former DA methods adopt some weight hyper-parameters to linearly combine the training objectives to form an overall objective. However, the gradient directions of these objectives may conflict with each other due to domain shift. Under such circumstances, the linear optimization scheme might decrease the overall objective value at the expense of damaging one of the training objectives, leading to restricted solutions. In this paper, we rethink the optimization scheme for DA from a gradient-based perspective. We propose a Pareto Domain Adaptation (ParetoDA) approach to control the overall optimization direction, aiming to cooperatively optimize all training objectives. Specifically, to reach a desirable solution on the target domain, we design a surrogate loss mimicking target classification. To improve target-prediction accuracy to support the mimicking, we propose a target-prediction refining mechanism which exploits domain labels via Bayes' theorem. On the other hand, since prior knowledge of weighting schemes for objectives is often unavailable to guide optimization to approach the optimal solution on the target domain, we propose a dynamic preference mechanism to dynamically guide our cooperative optimization by the gradient of the surrogate loss on a held-out unlabeled target dataset. Extensive experiments on image classification and semantic segmentation benchmarks demonstrate the effectiveness of ParetoDA