Blood Cell Detection

The LeukemiaAttri dataset is a large-scale, multi-domain collection of microscopy images derived from leukemia patient samples, enriched with detailed morphological information. This dataset comprises a total of 28.9K images (2.4K × 2 × 3 × 2), which were captured using both low-cost and high-cost microscopes at three different resolutions: 10x, 40x, and 100x, utilizing various cameras. In addition to providing location annotations for each white blood cell (WBC), the dataset includes comprehensive morphological attributes for every WBC, enhancing its utility for research and analysis in the field. | Sr. # | Dataset Name | Microscope | Resolution | Camera | Image Count | WBC Count | Artifacts Count | |-------|---------------|---------------|------------|----------------|-------------|-----------|-----------------| | 1 | H_100x_C1 | Olympus CX23 | 100x | Honor 9X Lite | 2.4k | 7.8k | 2.5k | | 2 | H_100x_C2 | Olympus CX23 | 100x | HD1500T | 2.4k | 7.8k | 2.5k | | 3 | H_40x_C1 | Olympus CX23 | 40x | Honor 9X Lite | 2.4k | 7.8k | 2.5k | | 4 | H_40x_C2 | Olympus CX23 | 40x | HD1500T | 2.4k | 7.8k | 2.5k | | 5 | H_10x_C1 | Olympus CX23 | 10x | Honor 9X Lite | 2.4k | 7.8k | 2.5k | | 6 | H_10x_C2 | Olympus CX23 | 10x | HD1500T | 2.4k | 7.8k | 2.5k | | 7 | L_100x_C1 | XSZ_107BN | 100x | Honor 9X Lite | 2.4k | 4.8k | 1.1k | | 8 | L_100x_C2 | XSZ_107BN | 100x | ZZCAT-5MP | 2.4k | 4.8k | 1.1k | | 9 | L_40x_C1 | XSZ_107BN | 40x | Honor 9X Lite | 2.4k | 7.8k | 2.5k | | 10 | L_40x_C2 | XSZ_107BN | 40x | ZZCAT-5MP | 2.4k | 7.8k | 2.5k | | 11 | L_10x_C1 | XSZ_107BN | 10x | Honor 9X Lite | 2.4k | 7.8k | 2.5k | | 12 | L_10x_C2 | XSZ_107BN | 10x | ZZCAT-5MP | 2.4k | 7.8k | 2.5k | | | Total Count | | | | 29k | 88.2k | 27.8k |

P. vivax (malaria) infected human blood smears with bounding box annotations. The data consists of two classes of uninfected cells (RBCs and leukocytes) and four classes of infected cells (gametocytes, rings, trophozoites, and schizonts).

Raw-Microscopy: 940 raw bright-field microscopy images of human blood smear slides for leukocyte classification (microscopy/images/raw_scale100) with corresponding labels (microscopy/labels). 5,640 variations measured at six additional different intensities (microscopy/images/raw_scale001-raw_scale0075) 11,280 images of the raw sensor data processed through twelve different pipelines (microscopy/images/processed_views) Raw-Drone: 548 raw drone camera images for car segmentation (drone/images_tiles_256/raw_scale100) with corresponding binary segmentation mask (drone/masks_tiles_256). The images and the masks are cropped from 12 raw drone camera images (drone/images_full/raw_scale100) and 12 masks (drone/masks_full) of size 3648 by 5472. 3,288 variations measured at six additional different intensities (drone/images_tiles_256/raw_scale001-raw_scale075). 6,576 images of the raw sensor data processed through twelve different pipelines (drone/images_tiles_256/processed_views). Detailed datasheets for the two datasets can be found in the appendices of https://arxiv.org/abs/2211.02578.

Microscopy is a cornerstone of biomedical research, enabling detailed study of biological structures at multiple scales. Advances in cryo-electron microscopy, high-throughput fluorescence microscopy, and whole-slide imaging allow the rapid generation of terabytes of image data, which are essential for fields such as cell biology, biomedical research, and pathology. These data span multiple scales, allowing researchers to examine atomic/molecular, subcellular/cellular, and cell/tissue-level structures with high precision. A crucial first step in microscopy analysis is interpreting and reasoning about the significance of image findings. This requires domain expertise and comprehensive knowledge of biology, normal/abnormal states, and the capabilities and limitations of microscopy techniques. Vision-language models (VLMs) offer a promising solution for large-scale biological image analysis, enhancing researchers’ efficiency, identifying new image biomarkers, and accelerating hypothesis generation and scientific discovery. However, there is a lack of standardized, diverse, and large-scale vision-language benchmarks to evaluate VLMs’ perception and cognition capabilities in biological image understanding. To address this gap, we introduce μ-Bench, an expert-curated benchmark encompassing 22 biomedical tasks across various scientific disciplines (biology, pathology), microscopy modalities (electron, fluorescence, light), scales (subcellular, cellular, tissue), and organisms in both normal and abnormal states. We evaluate state-of-the-art biomedical, pathology, and general VLMs on μ-Bench and find that: i) current models struggle in all categories, even for basic tasks such as distinguishing microscopy modalities; ii) current specialist models fine-tuned on biomedical data often perform worse than generalist models; iii) fine-tuning in specific microscopy domains can cause catastrophic forgetting, eroding prior biomedical knowledge encoded in their base model. iv) weight interpolation between fine-tuned and pre-trained models offers one solution to forgetting and improves general performance across biomedical tasks. We release μ-Bench under a permissive license to accelerate the research and development of microscopy foundation models.