Audioset is an audio event dataset, which consists of over 2M human-annotated 10-second video clips. These clips are collected from YouTube, therefore many of which are in poor-quality and contain multiple sound-sources. A hierarchical ontology of 632 event classes is employed to annotate these data, which means that the same sound could be annotated as different labels. For example, the sound of barking is annotated as Animal, Pets, and Dog. All the videos are split into Evaluation/Balanced-Train/Unbalanced-Train set. Source: Curriculum Audiovisual Learning

Consists of more than 210k videos for 310 audio classes. Source: VGGSound: A Large-scale Audio-Visual Dataset

The Few-Shot Object Learning (FewSOL) dataset can be used for object recognition with a few images per object. It contains 336 real-world objects with 9 RGB-D images per object from different views. Object segmentation masks, object poses and object attributes are provided. In addition, synthetic images generated using 330 3D object models are used to augment the dataset. FewSOL dataset can be used to study a set of few-shot object recognition problems such as classification, detection and segmentation, shape reconstruction, pose estimation, keypoint correspondences and attribute recognition. Motivation: If robots can recognize objects from a few exemplar images, it is possible to scale up the number of objects a robot can recognize because collecting a few images per object is a much easier process compared to building a 3D model of an object. In addition, models trained in the meta-learning setting can generalize to new objects without re-training.

We construct Gaze-CIFAR-10, a gaze-augmented image dataset based on the standard CIFAR-10 benchmark, enhanced with human eye-tracking annotations collected using the HTC VIVE Pro Eye headset. The original CIFAR-10 dataset consists of 60,000 color images across 10 categories, each with a resolution of $32 \times 32$ pixels. To enable reliable human gaze tracking, all images are upsampled to $1024 \times 1024$ using the Real-ESRGAN model. For each image, we collect a sequence of 176 eye-gaze coordinates, normalized to the range $[0, 224]$ (with the lower-left corner of the image as the origin) to match the input resolution of Vision Transformers (ViTs). Gaze data were collected from 20 participants using an HTC VIVE Pro Eye headset. The dataset contains 50,000 training images and 10,000 test images, with corresponding synchronized gaze trajectories.

We introduce GLAMI-1M: the largest multilingual image-text classification dataset and benchmark. The dataset contains images of fashion products with item descriptions, each in 1 of 13 languages. Categorization into 191 classes has high-quality annotations: all 100k images in the test set and 75% of the 1M training set were human-labeled. The paper presents baselines for image-text classification showing that the dataset presents a challenging fine-grained classification problem: The best scoring EmbraceNet model using both visual and textual features achieves 69.7% accuracy. Experiments with a modified Imagen model show the dataset is also suitable for image generation conditioned on text.

Mudestreda Multimodal Device State Recognition Dataset obtained from real industrial milling device with Time Series and Image Data for Classification, Regression, Anomaly Detection, Remaining Useful Life (RUL) estimation, Signal Drift measurement, Zero Shot Flank Took Wear, and Feature Engineering purposes. The official dataset used in the paper "Multimodal Isotropic Neural Architecture with Patch Embedding" ICONIP23. Official Minape repository: https://github.com/hubtru/Minape Official Mudestreda dataset: https://zenodo.org/records/8238653 Conference paper: https://link.springer.com/chapter/10.1007/978-981-99-8079-6_14 Mudestreda (MD) | Size 512 Samples (Instances, Observations)| Modalities 4 | Classes 3 | Future research: Regression, Remaining Useful Life (RUL) estimation, Signal Drift detection, Anomaly Detection, Multivariate Time Series Prediction, and Feature Engineering. ## Overview * Task: Uni/Multi-Modal Classification * Domain: Industrial Flank Tool Wear of the Milling Machine * Input (sample): 4 Images: 1 Tool Image, 3 Spectrograms (X, Y, Z axis) * Output: Machine state classes: Sharp, Used, Dulled * Evaluation: Accuracies, Precision, Recal, F1-score, ROC curve * Each tool's wear is categorized sequentially: Sharp → Used → Dulled. * The dataset includes measurements from ten tools: T1 to T10. * Data splitting options include random or chronological distribution, without shuffling. * Options: * Original data or Augmented data * Random distribution or Tool Distribution ## Use Cases: | Input | Model | Output | |-----------------|---------------|-----------------------| | Use Cases: | | | | 4 Images (1 Tool Image, 3 Spectrograms (X, Y, Z)) | Classification Model | Class (Flank Tool Wear: Sharp, Used, Dulled) | | 3 Spectrograms (X,Y,Z axis) | Classification Model | Class (Flank Tool Wear) | | 1 Tool Image | Classification Model | Image Class (Flank Tool Wear) | | Future Work: | | | | [1, ..., 4] Images | Model | Remaining Useful Life (RUL) estimation | | [1, ..., 4] Images | Monitoring Model | Fault and Anomaly Detection | | [1, ..., 4] Images | Forecasting Model | Multivariate Time Series Prediction | | [1, ..., 3] Spectrograms | Model | Signal Drift measurement | | [1, ..., 4] Images | Regression Model | Zero-Shot Flank Tool Wear (in µm, 10e-6 meter) | | [1, ..., 4] Images | Feature Engineering | Diagnostic Feature Designer If you use Mudestreda dataset cite the work Minape @ ICONIP2023. ## Cite the Paper: If you reference the papr or you use Mudestreda dataset cite the work Minape @ ICONIP2023 @inproceedings{truchan2023multimodal, title={Multimodal Isotropic Neural Architecture with Patch Embedding}, author={Truchan, Hubert and Naumov, Evgenii and Abedin, Rezaul and Palmer, Gregory and Ahmadi, Zahra}, booktitle={International Conference on Neural Information Processing}, pages={173--187}, year={2023}, organization={Springer} }