Few-shot fine-grained video action recognition requires recognizing novel action classes from only a few labeled examples. The difficulty is that visually similar actions often share actors, scenes, poses, and objects, differing only in the order, timing, and short duration of motion changes. We study this problem under three practical constraints: few tunable parameters, low compute cost, and use of the video modality alone. We propose Motion-aligned Temporal Adapter (MATA), a parameter-efficient adapter on top of a frozen CLIP backbone. MATA specializes CLIP features along the temporal dimension using two components: temporal phase alignment through learnable temporal positional embeddings, and a local motion transition operator through depth-wise convolution. With only ~3.6M tunable parameters, MATA achieves state-of-the-art performance across multiple fine-grained benchmarks while remaining robust to both visual corruptions and temporal disruptions induced by frame dropping. Code is available at https://github.com/prakashchhipa/MATA-Motion-aligned-Temporal-Adapter
Expert annotation in digital pathology is expensive, and low-label regimes remain common in realistic histopathology workflows. Self-supervised learning (SSL) offers a practical way to learn transferable representations from unlabeled pathology images, but performance-only evaluation can hide unreliable confidence behavior under domain shift. We present a reliability-aware benchmark of SimCLR, BYOL, and an MAE-inspired masked-reconstruction baseline on PatchCamelyon (PCam; 96x96) and PANDA (prostate cancer) under 1%, 5%, and 10% labeled regimes with frozen-encoder linear probing. To address common single-run concerns, ResNet-18 results are repeated over 3 splits x 3 seeds (n=9) and ViT-B/16 over 1 split x 3 seeds (n=3) under practical compute constraints, with mean+/-std reporting and per-run logs. Beyond AUROC, accuracy, and F1-score, we report Expected Calibration Error and Brier score, OOD confidence diagnostics for cross-dataset PCam<->PANDA scoring, federated-vs-centralized SimCLR comparisons on PCam, and cross-dataset transfer to colorectal histology tiles. SSL improves label efficiency on the more controlled PCam task, while PANDA's noisier patch-level proxy can remain near coin-flip with higher variance. The study highlights that histopathology SSL should be assessed not only by ranking performance but also by calibration, robustness, and transparent reproducibility diagnostics before clinical interpretation.
Car damage detection systems play a vital role in scenarios, like the insurance claims process, accident reporting systems, or a documented write-up of the various damages a vehicle in a car garage exhibits upon entry and exit. This paper presents a two-fold model architecture encompassing a convolutional neural network used to classify the collected input samples as clear or damaged, thereby filtering out the undamaged images, before the input is passed on to the object detection layer for further localization and classification of defects into Scratch, Dent, Glass Shatter, and Smash. A deviation from the focus of related work, this paper poses the use of unified models while applying transfer learning techniques to augment performance that caters to multiple use cases. The design of this model has been made to optimize performance by incorporating the two models to streamline the processing of input samples. The model boasts significant performance with accuracies of 98% for the CNN layer and 0.5 mAP for the object detection layer. However, the metric for the object detection layer seems to understate its performance due to reasons discussed within the paper.
Neural image compression (NIC) models are commonly evaluated with scalar rate-distortion metrics, which do not show how a codec treats individual spatial frequencies. We propose a benchmark that probes codecs with the DCT basis matrix and single-frequency test images. We measure relative off-target energy (leakage), the fraction of reconstruction energy outside the probed frequency, together with target-frequency gain. Exact decompositions and bounds relate these measurements to reconstruction error. At operating points nearest 1 bpp, the BMSHJ, MBT, and Cheng2020 models leave 80-97% of their reconstructed high-band energy outside the target bin and have target-frequency gains of 0.02-0.23, whereas TCM and FTIC attain off-target energy in the classical-codec range. Both probe types identify behavior absent from scalar metrics: released Cheng2020-Anchor models produce extreme pre-clamp outputs on isolated single-frequency inputs. Finally, decoder-only fine-tuning with natural-image and leakage losses reduces basis off-target energy by 90-99.8% across all eight evaluated architectures while keeping the bitstream fixed, at a Kodak PSNR cost of 0.09-0.59 dB.
The segmentation of brain tumors from magnetic resonance imaging (MRI) is an essential step for computer-aided neuro-oncology, treatment planning and follow-up. While effective encoder-decoder representations are achieved by U-Net and Attention U-Net, the skip sections of the network can generate weak-contrast or noisy responses in the case of heterogeneous boundaries of the tumor. In this paper, we introduce a Quality-Governed Deep Learning Network (QGDL-Net) that can be used for binary tumor segmentation from FLAIR MRI slices. The proposed model is able to predict the slice quality based on contrast and SNR statistics and to fine-tune the encoder features prior to decoding by learnable quality gates. We used the 80:20 train-validation split for experiments with BraTS 2021. QGDL-Net achieved a Dice of 0.9747, IoU of 0.9506, precision of 0.9755, recall of 0.9739, F1 of 0.9747 and accuracy of 0.9982, outperforming U-Net and Attention U-Net. The EigenCAM maps and segmentation overlays provide better tumor localization and interpretable activation around the lesion regions.
Diffusion-based anomaly detectors reconstruct a test image towards normality, yet how much of an anomaly survives, its leak, remains uncharacterized. We show that a first-order deterministic sampler obeys an exact recursion along the anomaly, whose continuous limit is a survival law. The anomaly decays exponentially as the noise falls, at a rate set by the retention $r$, the share the model keeps on the states it visits. Because $r$ depends on the noise level and not on the step count, one measured path predicts the leak of other step counts, samplers and schedules from a nearby start to within a few hundredths, on four diffusion models and real defects. The law also shows that the leak's growth over steps is exposure bias and that normal details obey the same law, so changing the path or the training trades leak for detail error. We introduce SurvAD, which fine-tunes the reconstructor on inference-like starts and fits a comparator that adds evidence to the residual map without suppressing its ranking elsewhere. For under $2\%$ more inference time, SurvAD raises the multi-class pixel average precision (P-AP) of DeCo-Diff from $76.8$ to $80.7$ on MVTec AD and from $57.5$ to $60.0$ on VisA. It matches CostFilter-AD on the same reconstructor with $9\times$ fewer trainable parameters and, under Dinomaly's protocol, exceeds its reported P-AP on Dinomaly on both benchmarks.
Associative memory models classically describe retrieval as convergence to stored patterns, but the theory of sequential retrieval remains comparatively limited. We develop a two-timescale input-driven Hopfield network in which fast associative retrieval is modulated by a slow feedback variable evolving over a prescribed memory-transition graph. This yields autonomous, graph-constrained transitions while preserving analytical tractability. Using slow-fast and geometric singular perturbation theory, we characterize memory fixation via memory manifolds and identify the geometric regions that govern stability loss and escape from retrieved memories. We further derive reduced slow dynamics and explicit criteria for transition onset, self-sustained retrieval, and collapse, expressed through escape times, gain thresholds, and fixed points. Together, these results provide a tractable framework for analyzing feedback-driven sequential retrieval in Hopfield networks.
Peptide sequence-property relationships are often highly non-linear, but can be learned with machine learning models and exploited for the design of new materials and bioactive compounds. Various methods based on transformer architectures have become available for molecular property predictions. These foundation models promise straightforward inference and prediction of properties of new materials through in-context learning. Benchmarks of individual model architectures exist, but a systematic comparison of various foundation models against classical baselines on similar datasets is lacking. Here, we provide a systematic evaluation of the predictive performance of various in-context learning strategies. We compare several large language models, tabular foundation models, and typed decision models against an XGBoost baseline. We evaluate predictive performance on binary and multiclass classification tasks on pH-dependent surface activ-ity, IR spectral signatures, self-assembly, and anti-cancer activities. Our results show that ICL is competitive, yet does not significantly outperform classical baselines, and performance varies between models and tasks. Our findings further suggest that language models use both in context examples and knowledge acquired during pretraining to infer properties of unseen samples. This finding indicates that pretrained models can leverage the existing literature for predictions, but may also bias their predictions towards established sequence-property relationships.
Chest radiograph report generation is hampered by retrieved reports that resemble the current image but are clinically inconsistent with it, and by diagnostic-label imbalance. We present RG-MVR, a retrieval-grounded framework comprising Retrieval-Augmented Evidence Grounding, which combines dense memory summaries with visually gated sparse retrieved evidence; Adaptive Multi-view Evidential Reasoning, which weights image, memory, and retrieved evidence per case; and a Clinical Prompt Learner, which models finding mention separately from positive, negative, or uncertain status. On MIMIC-CXR, RG-MVR achieved a clinical-label F1 of 0.483 and a BLEU-4 of 0.118 (IU X-ray: 0.202 and 0.110). In an exploratory external cohort of 22 examinations, clinical-label agreement declined markedly (F1, 0.071) and unsupported statements remained common, although physicians found fewer unsupported temporal comparisons (9 vs. 20 cases) and device-related statements (7 vs. 15) than with the baseline. Evidence-aware retrieval and structured diagnostic prompting may reduce specific reporting errors, but broader external validation is required.
Open Radio Access Network (O-RAN) improves radio access network flexibility through disaggregation, open interfaces, and programmable RAN Intelligent Controllers (RICs). However, the same openness and distribution increase the attack surface and make centralized intrusion detection difficult to scale across heterogeneous RAN sites. This paper presents a federated lightweight Transformer framework for privacy-preserving network-layer cyberattack detection in O-RAN. Each participating site trains a compact Transformer locally on temporally ordered network-flow records, while only model parameters are shared with a federated coordinator. To model realistic site heterogeneity, the client datasets are partitioned according to a Dirichlet distribution, resulting in non-independent and identically distributed (non-IID) local data. The local detector uses feature projection, positional encoding, multi-head self-attention, feed-forward layers, and temporal pooling to classify five network-layer attack categories: ICMP, Slowloris, SYN, TCP-ACK, and UDP. Experiments on the NetsLab 5G O-RAN intrusion-detection dataset show an overall accuracy of 94.89 % and a macro-F1 score of 81.09 %.
Multi-agent LLM systems often reuse the same agents across tasks, creating an opportunity to draw on past interactions when deciding whom to consult. We study this form of memory with PeerMem, which saves MACE's LinUCB state and reuses it when the same agents return. Using six agents, a fixed set of 80 HotpotQA questions, and 12 paired lineages per model, we compare cold, warm, and shuffled initialization in an answer-exchange protocol with reference-answer feedback. Under stable evidence access, warm memory lowers pooled hindsight regret and increases consultation of other evidence holders on Qwen3-32B and Gemma-4-31B-it, while final token F1 differences remain uncertain. Qwen interaction traces further show that warm agents more often encounter peer answers that score above their own prior answers. We extend the evaluation to Qwen3.5-9B using freshly learned memory to compare warm and cold initialization under reassigned and paired stable roles. The primary changed-minus-stable final-F1 interaction is -1.38 points, with a 95% interval of [-3.50,+0.73]. Together, these experiments trace the effects of peer memory through consultation, answer revision, and final task performance.
Vision Transformers (ViTs) perform strongly in face recognition (FR), yet their fixed-depth inference limits deployment in resource-constrained settings. We propose SD-ViT, a training framework that gives a usable exit at every block depth by making every depth produce discriminative embeddings, for budget-selected fixed-depth exits. Standard ViTs supervise intermediate layers only through backpropagation, so their intermediate representations are not directly optimized to form usable identity embeddings. SD-ViT adds a stochastic auxiliary signal: at each iteration a randomly selected intermediate block is passed through the same embedding head as the final layer and trained with a weighted margin-penalty softmax loss. The objective is thus stochastically sampled deep supervision through a shared classifier. Because both loss terms share that head and classifier, intermediate tokens are trained to map into the same embedding space as final-layer tokens. This matters for FR, where matching is by similarity: a probe processed at an early exit can be compared directly against full-depth gallery embeddings, with no gallery recomputation or adaptation layers. On established benchmarks, SD-ViT stays within 0.4 IJB-C points of the same backbone trained conventionally at full depth, while its early-exit probes recover over 96% of their own full-depth performance at 50% of the computational cost without modifying the reference gallery. A single verification threshold calibrated at full depth remains usable from quarter depth onward.
Inference-time steering in vision-language models often entangles query-relevant reasoning dynamics with low-level perceptual noise, causing collateral degradation on correct predictions. We propose Question-Contrastive Visual Steering (QCVS) to isolate task-specific intervention directions via double-differential activation probing. By contrasting spatial layout perturbations under the target query against a generic description reference, QCVS cancels task-agnostic visual artifacts without ground-truth supervision. A dual-threshold gating mechanism then evaluates relative interaction strength and signal amplitude, selectively applying a single bounded hidden-state update only when necessary. Empirical evaluations across multiple scales of Qwen3.5 (2B to 9B) and DeepSeek-VL2 on DynaMath and We-Math demonstrate that QCVS achieves state-of-the-art accuracy gains under sparse coverage (approximately 22%-30%). Crucially, QCVS yields repair rates up to 13.98% while suppressing collateral damage down to 0.41%, proving high intervention precision and computational efficiency.
We introduce P-IMLA, a preconditioned implicit-midpoint Langevin sampler for high-dimensional log-concave targets with non-smooth regularization. The obstruction is that the large steps enabled by matrix preconditioning are precisely where preconditioned Euler-Maruyama inflates stationary variance. Implicit midpoint removes this surplus via a Cayley-transform identity, making P-IMLA Gaussian-exact at any step size. Beyond Gaussians, whitening transfers the sharp IMLA contraction rate to $\kappa_{\mathrm{eff}}$, and backward-error analysis gives drift-only rather than Euler-type diffusion bias under explicit smoothness assumptions. An $M$-norm Moreau envelope yields a proximal non-smooth variant with a three-term error budget. Controlled Gaussian and TV-regularized imaging experiments confirm the predicted separation: the unpreconditioned midpoint scheme retains long transients as $\kappa_f$ grows, whereas P-IMLA remains $\kappa_{\mathrm{eff}}$-governed and near variance-exact.
Language models are widely used as synthetic respondents in surveys, behavioral experiments, and market research. Most of these respondents read text, whereas shoppers in a store choose by looking at shelves, which makes vision-language models (VLMs) natural synthetic shoppers. We test whether VLM shoppers follow real consumers at two levels. At the purchase level, VLM shoppers built from household records plan their trips from a store directory and select products from shelf images, and we compare their purchases with six public datasets from three countries. The synthetic baskets hit held-out purchases as often as the households' own past trips, and synthetic group differences correlate with real ones twice as strongly as those of a trained regression. Without fitting, VLM shoppers reproduce alcohol purchases by gender and age in Russia and trip sizes by store distance in Taiwan, and respond to situations, prices, and displays within measured ranges. At the level of a single shelf choice, a model's answer confounds the product with its position and letter label. We therefore introduce ShelfSight, shelf renderings that vary products, positions, labels, viewpoints, and prices independently, together with a factor model that separates product preferences from the effects of label and position, which we call answer effects. On ShelfSight, hosted VLMs read the shelf, follow stated needs, keep their product preferences across views, and respond to price as budget shoppers. These answer effects, a preference for the letter A and for positions in the image, are estimated once per image type and removed. VLMs are thus good synthetic shoppers whose corrected shelf choices reflect product preferences.
Autoregressive video diffusion models produce large KV caches during multi-round video generation. Keeping all these KV chunks would consume substantial memory, while rebuilding discarded caches would significantly delay video chunk generation. This paper presents ReelCache, a KV cache compression system for efficient multi-round, long video generation. ReelCache combines an intra-chunk compressor with an inter-chunk cache manager. The compressor applies low-rank decomposition across channels and video coding to the resulting coefficients, predicting the decomposition rank per transformer layer online to meet retained-energy targets. The cache manager shares basis directions across consecutive chunks and organizes chunk states in a tree, allowing generation branches to share historical state while supporting restoration for resume, rewind, and fork. Experiments across five video generation models show that ReelCache reduces KV storage by 14.46-21.13x relative to baselines and accelerates next-chunk generation by 1.25-3.36x, with less than 1% drop in six representative VBench metrics. GitHub: https://github.com/Tianyu9748/ReelCache
Objective: It is not known whether the input representation governs the quality of explanations of deep learning models for electrocardiogram analysis. Risk of dangerous ventricular repolarisation is judged from the QT-interval and the shape of the T-wave, features that are hard to assess visually and harder still to verify in a model's reasoning. We evaluate a pseudo-colour encoding filling the area beneath the QT interval, its colour sequence designed from clinical electrophysiology knowledge and mapped to a validated risk prediction model, the QT-nomogram. Methods: Two identically configured residual networks were trained on single-heartbeat images from a clinical trial, differing only in whether pseudo-colour encoding was applied, so that any difference is attributable to the chromatic channel alone. To externally validate the effect, both models were applied without fine-tuning or threshold tuning to a previously unseen epilepsy cohort from the UK-Biobank at two visits, and eight explanation methods compared on faithfulness, localisation and stability. Results: The pseudo-colour model achieved areas under the receiver operating characteristic curve of 0.903 and 0.914 at the first and follow-up visits, against 0.719 and 0.720 for the monochrome model, recovering three to five times as many at-risk heartbeats at comparable specificity. Pseudo-colour encoding improved explanation localisation and stability for almost every method, and the improvement reproduced across both visits. Conclusion: The input representation is a controllable and previously overlooked determinant of both predictive performance and explanation quality. Significance: Treating interpretability as a representation-design problem offers a route towards explanations clinicians can check against known electrophysiology.
Although audio-driven human animation has achieved impressive realism, it still lacks effective Multi-Scene Temporal Control: orchestrating background transitions at precise timestamps while preserving a coherent foreground subject. We show that this challenge stems from the diffusion transformer's self-attention mechanism, and addressing it requires two capabilities: (1) Semantic-Aware Temporal Decoupling, which isolates background attention across scenes while retaining cross-scene foreground attention for subject consistency; and (2) Foreground Motion Localization, which accurately tracks the foreground subject across latent frames, especially under large motion. To address these challenges, we introduce SceneShifter, a training-free framework for precise multi-scene control in audio-driven human animation. SceneShifter guides spatio-temporal self-attention by suppressing cross-scene attention among background tokens while preserving cross-scene attention among foreground tokens. To support this guidance under large motion, SceneShifter extracts dynamic foreground masks from the self-attention layers and heads that best capture subject trajectories. We further introduce SceneShifterBench, a benchmark designed to evaluate scene-timing accuracy, foreground preservation, and visual quality under large motion, multiple subjects. Experiments show that SceneShifter achieves frame-level scene timing, strong subject preservation, and high visual quality, outperforming existing baselines. Code is available at https://github.com/JamesLong199/SceneShifter.
This note describes a multi-view video dataset of a deformable object driven by a measured input, with the measured pose of every camera, and a first 3D result obtained on it with LaGSplat. LaGSplat learns from video a low-dimensional latent state that is both the generalized coordinate of a learned dissipative Lagrangian and the conditioning variable of a Gaussian Splatting decoder; fitting the decoder in 3D calls for synchronized multi-view footage, which the original paper only had from synthetic renderings. Here a robot arm moves a flexible strip by oscillating along a one-dimensional path, and four iPhones, three fixed and one handheld, film it for ten minutes. The videos are synchronized, the cameras are posed from the scene, and the position of the gripper is given per frame as the input of the system. LaGSplat fitted on this data with the gripper acceleration as a forcing term renders held-out views at 26 to 33 dB PSNR and runs as a real-time interactive demo in a web browser. Dataset, checkpoints and training code are public.
Hippocampal atrophy is among the earliest structural markers of Alzheimer's disease, and reliable hippocampal segmentation across serial MRI visits is a prerequisite for measuring it. We present HippoMamba, a network that combines a 3D convolutional encoder and decoder with a Mamba selective state space block at the bottleneck, designed to process one or more MRI visits of a subject. On the Medical Segmentation Decathlon hippocampus dataset (260 subjects, five-fold cross-validation), HippoMamba obtained a mean Dice similarity coefficient of 0.8861 $\pm$ 0.0094, the highest among six convolutional and transformer-based methods trained with the same folds and patch size, and its per-subject Dice was significantly higher than that of each method (Holm-corrected Wilcoxon tests, $p \leq 0.013$). Its mean 95th percentile Hausdorff distance (1.45 mm) and average surface distance (0.44 mm) were the lowest of the compared methods, although they did not differ significantly from those of Attention U-Net, and the Hausdorff distance did not differ significantly from that of SegResNet. Without retraining, the five-fold ensemble was applied to 600 T1-weighted sessions from 150 participants of the OASIS-3 cohort. In sessions meeting a registration quality criterion, predicted hippocampal volumes ranked consistently with FreeSurfer volumes (Pearson r = 0.778, 95% CI 0.699 to 0.838; ICC(3,1) = 0.772) but were systematically smaller by about 27%. In a pre-specified analysis of participants followed for at least two years, annualised volume change derived from per-visit segmentation was moderately correlated with that of FreeSurfer (r = 0.503, 95% CI 0.319 to 0.659), with a larger mean rate. These results support the use of HippoMamba for hippocampal segmentation and for ranking hippocampal volume within a cohort, with absolute volumes and rates of change interpreted relative to the reference method.